[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482623#comment-17482623
]
Caleb Rackliffe edited comment on CASSANDRA-17188 at 1/26/22, 5:12 PM:
---
[~benedict] So what does this mean, in concrete terms? Is your ask essentially
that we come up with at least a _design_ for CASSANDRA-17292 (i.e. a design we
broadly agree on as a long term guide, but don't need to actually complete
until 5.0 at the earliest) and then make a reasonable effort to have guardrails
fit into that for 4.1 (where it will be released for the first time and not
create compatibility burdens)? If we can do that incrementally, I'm not opposed
to that strategy, assuming we actually want to move the config in that
direction in our lifetimes :)
[~adelapena] [~dcapwell] Thoughts?
(Again, I think CASSANDRA-17148 is where the rubber meets the road on this.)
was (Author: maedhroz):
[~benedict] So what does this mean, in concrete terms? Is your ask essentially
that we come up with at least a _design_ for CASSANDRA-17292 (i.e. a design we
broadly agree on as a long term guide, but don't need to actually complete
until 5.0 at the earliest) and then make a reasonable effort to have guardrails
fit into that for 4.1 (where it will be released for the first time and not
create compatibility burdens)? If we can do that incrementally, I'm not opposed
to that strategy, assuming we actually want to move the config in that
direction in our lifetimes :)
[~adelapena] [~dcapwell] Thoughts?
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482612#comment-17482612
]
Benedict Elliott Smith edited comment on CASSANDRA-17188 at 1/26/22, 4:55 PM:
--
bq. That's why any potential rearrangement of the config should be easier to do
with guardrails than with the current scattered properties.
You seem to be missing my point. You have already created changes to the config
file, introducing a concept that relates to existing concepts in the config
file. By doing so you have created an inconsistency. You are suggesting this is
fine, because you think it is unimportant and somebody can fix it later. I am
saying it is not fine, and I want you to address it now. I am saying this
because that is, in my view, how software is properly developed: you address
problems as you introduce them.
was (Author: benedict):
bq. That's why any potential rearrangement of the config should be easier to do
with guardrails than with the current scattered properties.
You seem to be missing my point. You have already created changes to the config
file, introducing a concept that relates to existing concepts in the config
file. By doing so you have created an inconsistency. You are suggesting this is
fine, because you think it is unimportant and somebody can fix it later. I am
saying it is not fine, and I want you to address it now. I am saying this
because that is, in my view, how software is properly developed: you address
problems as you introduce them. Otherwise you are creating work for others.
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17482612#comment-17482612
]
Benedict Elliott Smith edited comment on CASSANDRA-17188 at 1/26/22, 4:47 PM:
--
bq. That's why any potential rearrangement of the config should be easier to do
with guardrails than with the current scattered properties.
You seem to be missing my point. You have already created changes to the config
file, introducing a concept that relates to existing concepts in the config
file. By doing so you have created an inconsistency. You are suggesting this is
fine, because you think it is unimportant and somebody can fix it later. I am
saying it is not fine, and I want you to address it now. I am saying this
because that is, in my view, how software is properly developed: you address
problems as you introduce them. Otherwise you are creating work for others.
was (Author: benedict):
bq. That's why any potential rearrangement of the config should be easier to do
with guardrails than with the current scattered properties.
You seem to be missing my point. You have already created changes to the config
file, introducing a concept that relates to existing concepts in the config
file. By doing so you have created an inconsistency. You are suggesting this is
fine, because you think it is unimportant and somebody can fix it later. I am
saying it is not fine, and I want you to address it now.
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480166#comment-17480166
]
Caleb Rackliffe edited comment on CASSANDRA-17188 at 1/21/22, 4:24 PM:
---
[~benedict] Aside from this, I still think we (more than happy to do it myself)
should create a new Jira that _does_ start to socialize our overall proposal(s)
for the config as soon as CASSANDRA-15234 merges. Even if we intend for that to
be a 5.0 item, it can inform the guardrails/limits work here before it gets
release in 4.1, which is the worry I'm sure you and I both have around this.
was (Author: maedhroz):
[~benedict] Aside from this, I still think we (more than happy to do it myself)
should create a new Jira that _does_ start to socialize our overall proposal(s)
for the config as soon as CASSANDRA-15234 merges, even if we intend for that to
be a 5.0 item.
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480136#comment-17480136
]
Josh McKenzie edited comment on CASSANDRA-17188 at 1/21/22, 3:28 PM:
-
Thanks for that clarifying context Benedict.
{quote}Ideally, we would have design documents
{quote}
Hold on now, let's not get too wild here. ;)
What's the perceived work overhead / tension of refactoring whatever config
gets merged here w/regards to CASSANDRA-17212 and CASSANDRA-15234 if it hits
before they do, or the overhead of rebasing this on top of whatever we settle
on there authoritatively long-term if we move forward on all still and they
merge first?
ISTM the real risk here is this patch making it in before a major and
CASSANDRA-15234 _not_ making it leading to us releasing with a non-agreed upon
long-term config API commitment right? Perhaps the right choice is to just make
this ticket dependent on CASSANDRA-15234 and block merge of this on that so we
don't have a release get out with an API that's not widely agreed upon. We
could push this to "completion" and rebase it (sorry [~adelapena]) on top of
that one once it hits but more or less be ready to merge.
Also, in all seriousness, config parameters and formatting are (IMO) 100% an
API choice that has huge impacts on end users; it's one of the primary ways
operators interact with the system. We should think this out and have a design
doc where we talk through things high level instead of trawling through the mud
and trenches while trying to reason about it (like we so often do =/).
I'm operating under the assumption that's something we're going to
authoritatively determine separately and the work here will be deferring to
that in its final form.
was (Author: jmckenzie):
Thanks for that clarifying context Benedict.
{quote}Ideally, we would have design documents
{quote}
Hold on now, let's not get crazy here. ;)
What's the perceived work overhead / tension of refactoring whatever config
gets merged here w/regards to CASSANDRA-17212 and CASSANDRA-15234 if it hits
before they do, or the overhead of rebasing this on top of whatever we settle
on there authoritatively long-term if we move forward on all still and they
merge first?
ISTM the real risk here is this patch making it in before a major and
CASSANDRA-15234 _not_ making it leading to us releasing with a non-agreed upon
long-term config API commitment right? Perhaps the right choice is to just make
this ticket dependent on CASSANDRA-15234 and block merge of this on that so we
don't have a release get out with an API that's not widely agreed upon. We
could push this to "completion" and rebase it (sorry [~adelapena]) on top of
that one once it hits but more or less be ready to merge.
Also, in all seriousness, config parameters and formatting are (IMO) 100% an
API choice that has huge impacts on end users; it's one of the primary ways
operators interact with the system. We should think this out and have a design
doc where we talk through things high level instead of trawling through the mud
and trenches while trying to reason about it (like we so often do =/).
I'm operating under the assumption that's something we're going to
authoritatively determine separately and the work here will be deferring to
that in its final form.
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17480078#comment-17480078
]
Benedict Elliott Smith edited comment on CASSANDRA-17188 at 1/21/22, 1:46 PM:
--
What is a "flavor" of configuration? No coherent design for the configuration
file, and how it relates to existing context, has been presented for this epic.
This should be basic hygiene for API design in a mature project, just like
testing and QA. Once such a design has been proposed, alternatives (such as the
one I favour) can be weighed up, but right now we're talking about favouring
uncoordinated changes that consider only the change in question and none of the
broader context - even the future work part of the same declared epic. That is
really bad hygiene, whatever "flavor" you prefer.
But I anyway strongly dislike the idea of dismissing API design considerations
as flavours. There are tensions in all API design, and they need to be
discussed. While there may be people who prefer one approach over another, this
is true even for deeply technical topics, and it does not make it any less an
important consideration. Dismissing these tensions is lazy, and lands us in a
mess of incoherent design like we are addressing in CASSANDRA-15234, which is
only one part of the codebase.
was (Author: benedict):
What is a "flavor" of configuration? No coherent design for the configuration
file, and how it relates to existing context, has been presented for this epic.
This should be basic hygiene for API design in a mature project, just like
testing and QA. Once such a design has been proposed, alternatives (such as the
one I favour) can be weighed up, but right now we're talking about favouring
uncoordinated changes that consider only the change in question and none of the
broader context - even the future work part of the same declared epic. That is
really bad hygiene, whatever "flavor" you prefer.
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17479541#comment-17479541
]
Caleb Rackliffe edited comment on CASSANDRA-17188 at 1/20/22, 5:39 PM:
---
Perhaps we should resolve the ongoing conversation around config structure in
CASSANDRA-17212 (which actually started in CASSANDRA-15234 before that was
focused on just human-readable parameter names) before we press forward w/ this?
was (Author: maedhroz):
Perhaps we should resolve the ongoing conversation around config structure in
CASSANDRA-17212 before we press forward w/ this?
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17188) Guardrails for consistency levels
[
https://issues.apache.org/jira/browse/CASSANDRA-17188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17472671#comment-17472671
]
Andres de la Peña edited comment on CASSANDRA-17188 at 1/11/22, 12:13 PM:
--
The proposed patch adds guardrails to warn about or reject read/write
consistency levels:
||PR||CI||
|[trunk|https://github.com/apache/cassandra/pull/1392]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1232/workflows/e733629c-3559-4999-9ea7-14b71bc30c2c]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1232/workflows/4a59214a-abd9-40a5-a134-7cc05741ae55]|
The part for write consistency levels is based on a previous patch written by
[~Ge].
Some queries can trigger internal queries that use a different consistency
level than that specified for the user. For example, certain write operations
on lists can [trigger a
read|https://github.com/adelapena/cassandra/blob/bc20bddcebd6a37b14cfbdd50c359be4c9743f73/src/java/org/apache/cassandra/cql3/statements/ModificationStatement.java#L396].
As another example, the different phases of Paxos in LWT can [internally use
multiple consistency
levels|https://github.com/adelapena/cassandra/blob/bc20bddcebd6a37b14cfbdd50c359be4c9743f73/src/java/org/apache/cassandra/service/StorageProxy.java#L445-L447].
These internal consistency levels are not verified by the proposed guardrails,
which are only applied to the regular and serial consistency levels explicitly
specified by the user.
The patch modifies [the existing {{Values}} type of
guardrail|https://github.com/adelapena/cassandra/blob/bc20bddcebd6a37b14cfbdd50c359be4c9743f73/src/java/org/apache/cassandra/db/guardrails/Values.java#L40-L41]
to have [a list of {{warned}}
values|https://github.com/adelapena/cassandra/blob/17188-trunk/src/java/org/apache/cassandra/db/guardrails/Values.java#L41-L43].
I think that this set of {{warned}} values can also be useful for the already
existing guardrail for table properties. I have modified that guardrail to also
include warned properties, in addition to the current sets of ignored and
disallowed properties:
{code:java}
# Guardrail to warn about, ignore or reject properties when creating tables.
table_properties:
warned: []
ignored: []
disallowed: []
{code}
was (Author: adelapena):
The proposed patch adds guardrails to warn about or reject read/write
consistency levels:
||PR||CI||
|[trunk|https://github.com/apache/cassandra/pull/1392]|[j8|https://app.circleci.com/pipelines/github/adelapena/cassandra/1232/workflows/e733629c-3559-4999-9ea7-14b71bc30c2c]
[j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/1232/workflows/4a59214a-abd9-40a5-a134-7cc05741ae55]|
The part for write consistency levels is based on a previous patch written by
[~Ge].
Some queries can trigger internal queries that use a different consistency
level than that specified for the user. For example, certain write operations
on lists can trigger a read. As another example, the different phases of Paxos
in LWT can internally use multiple consistency levels. These internal
consistency levels are not verified by the proposed guardrails, which are only
applied to the regular and serial consistency levels explicitly specified by
the user.
The patch modifies the existing {{Values}} type of guardrail to have a list of
{{warned}} values. I think that this set of {{warned}} values can also be
useful for the already existing guardrail for table properties. I have modified
that guardrail to include warned properties, in addition to the current ignored
and disallowed properties:
{code:java}
# Guardrail to warn about, ignore or reject properties when creating tables.
table_properties:
warned: []
ignored: []
disallowed: []
{code}
> Guardrails for consistency levels
> -
>
> Key: CASSANDRA-17188
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17188
> Project: Cassandra
> Issue Type: New Feature
> Components: Feature/Guardrails
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.x
>
>
> Add guardrails for read/write consistency levels, for example:
> {code:java}
> # Guardrail to warn about or reject read consistency levels.
> # By default all consistency levels are allowed.
> read_consistency_levels:
> warned: []
> disallowed: []
> # Guardrail to warn about or reject write consistency levels.
> # By default all consistency levels are allowed.
> write_consistency_levels:
> warned: []
> disallowed: []
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits
