[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487627#comment-17487627
]
Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 2/6/22, 6:12 AM:
--
I received an email that travis has failed for cassandra-dtest. It shows me
that it failed with syntax error during execution pytest --metatests. I pulled
the latest ccm locally and reran that step. It completed fine, same with my
branch.
In the meantime I saw that I missed to refert to riptano/ccm in
requirements.txt. It seems it wasn't in the separate branch as I thought. I
just ninja fixed that and I am going to see whether travis will fail again.
Jenkins is still running, nothing unusual there for now.
was (Author: e.dimitrova):
I received an email that travis has failed for cassandra-dtest. It shows me
that it failed with syntax error during execution pytest --metatests. I pulled
the latest ccm locally and reran that step. It completed fine.
In the meantime I saw that I missed to refert to riptano/ccm in
requirements.txt. It seems it wasn't in the separate branch as I thought. I
just ninja fixed that and I am going to see whether travis will fail again.
Jenkins is still running, nothing unusual there for now.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487627#comment-17487627
]
Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 2/6/22, 6:12 AM:
--
I received an email that travis has failed for cassandra-dtest. It shows me
that it failed with syntax error during execution pytest --metatests. I pulled
the latest ccm locally and reran that step. It completed fine, same with my
branch.
In the meantime I saw that I missed to refert to riptano/ccm in
requirements.txt. It seems it wasn't in the separate commit as I thought. I
just ninja fixed that and I am going to see whether travis will fail again.
Jenkins is still running, nothing unusual there for now.
was (Author: e.dimitrova):
I received an email that travis has failed for cassandra-dtest. It shows me
that it failed with syntax error during execution pytest --metatests. I pulled
the latest ccm locally and reran that step. It completed fine, same with my
branch.
In the meantime I saw that I missed to refert to riptano/ccm in
requirements.txt. It seems it wasn't in the separate branch as I thought. I
just ninja fixed that and I am going to see whether travis will fail again.
Jenkins is still running, nothing unusual there for now.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487629#comment-17487629
]
Ekaterina Dimitrova edited comment on CASSANDRA-15234 at 2/6/22, 6:11 AM:
--
Jenkins failed and I found in the logs the same error. I just restarted the
build... Considering that in travis things completed fine now, maybe here the
same will happen. I am totally not sure where this came from... I will keep an
eye on it
was (Author: e.dimitrova):
Jenkins failed and I found in the logs the same error. I just restarted the
build... Considering that in travis things completed fine now, maybe here the
same will happen. I am totally not sure where this came from... I will keep an
eye
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487629#comment-17487629
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
Jenkins failed and I found in the logs the same error. I just restarted the
build... Considering that in travis things completed fine now, maybe here the
same will happen. I am totally not sure where this came from... I will keep an
eye
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487628#comment-17487628
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
Ok, travis just completed successfully after the ninja fix. I have no clue why
was that error on a line in a file I haven't touched.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487627#comment-17487627
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
I received an email that travis has failed for cassandra-dtest. It shows me
that it failed with syntax error during execution pytest --metatests. I pulled
the latest ccm locally and reran that step. It completed fine.
In the meantime I saw that I missed to refert to riptano/ccm in
requirements.txt. It seems it wasn't in the separate branch as I thought. I
just ninja fixed that and I am going to see whether travis will fail again.
Jenkins is still running, nothing unusual there for now.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch trunk updated: ninja fix
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git The following commit(s) were added to refs/heads/trunk by this push: new 517cb17 ninja fix 517cb17 is described below commit 517cb17bc6c114ca3a9d3ab9c25efaf364e81550 Author: Ekaterina Dimitrova AuthorDate: Sun Feb 6 00:25:19 2022 -0500 ninja fix --- requirements.txt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/requirements.txt b/requirements.txt index cbf6ec5..d7c396a 100644 --- a/requirements.txt +++ b/requirements.txt @@ -9,8 +9,7 @@ # # In case you want to test a patch with your own CCM branch, further to changing below CCM repo and branch name, you need to add -e flag at the beginning # Example: -e git+https://github.com/userb/ccm.git@cassandra-17182#egg=ccm -# git+https://github.com/riptano/ccm.git@cassandra-test#egg=ccm --e git+https://github.com/ekaterinadimitrova2/ccm.git@CASSANDRA-15234-3#egg=ccm +git+https://github.com/riptano/ccm.git@cassandra-test#egg=ccm decorator docopt enum34 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Test and Documentation Plan:
CCM patch committed
[here|https://github.com/riptano/ccm/commit/6d676896fb2ced4b6ac62fdba4ae2570351b38df]
and retagged.
DTest patch committed
[here|https://github.com/apache/cassandra-dtest/commit/3935906a685640b2f6a2058b38fdf45d917edfc9]
Trunk patch split to 13 patches starts from [this
one|https://github.com/apache/cassandra/commit/db9f7a67ec4b03413c10034956e2cf18739ca4b1]
on.
The existing tests plus new unit tests added. Also, dtests exercise the
backward compatibility and test that way that ccm supports the same behavior as
Cassandra as regards to configuration parameters loading.
was:
[trunk|https://github.com/apache/cassandra/pull/1350] |
[dtest|https://github.com/apache/cassandra-dtest/pull/169] |
[ccm|https://github.com/ekaterinadimitrova2/ccm/pull/1]
The existing tests plus new unit tests added. Also, dtests exercise the
backward compatibility and test that way that ccm supports the same behavior as
Cassandra as regards to configuration parameters loading.
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Fix Version/s: 4.1
(was: 4.x)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Since Version: 4.0.1
Source Control Link:
https://github.com/apache/cassandra/commit/db9f7a67ec4b03413c10034956e2cf18739ca4b1
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Since Version: (was: 4.0.1)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-15234:
Status: Ready to Commit (was: Review In Progress)
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.x
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[
https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487621#comment-17487621
]
Ekaterina Dimitrova commented on CASSANDRA-15234:
-
Approvals taken on GitHub and confirmed in Slack.
Many rounds of testing on the different commits were executed, but this is the
final CI:
[j8|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1337/workflows/38632935-e720-4f5a-85d7-570fc52dbbee]
[j11|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/1337/workflows/16bfc433-02fc-4164-ae3a-9d66c577c708]
All issues have tickets except the one test that failed on Java 11 but it is
some issue with port already occupied which I've seen also with other tests and
runs and it is infra related.
CCM patch committed
[here|https://github.com/riptano/ccm/commit/6d676896fb2ced4b6ac62fdba4ae2570351b38df]
and retagged.
DTest patch committed
[here|https://github.com/apache/cassandra-dtest/commit/3935906a685640b2f6a2058b38fdf45d917edfc9]
Trunk patch split to 13 patches starts from [this
one|https://github.com/apache/cassandra/commit/db9f7a67ec4b03413c10034956e2cf18739ca4b1]
on.
Thank you and please don't hate me if Jenkins get down after seeing so many
commits :D
> Standardise config and JVM parameters
> -
>
> Key: CASSANDRA-15234
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15234
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Benedict Elliott Smith
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.x
>
> Attachments: CASSANDRA-15234-3-DTests-JAVA8.txt
>
>
> We have a bunch of inconsistent names and config patterns in the codebase,
> both from the yams and JVM properties. It would be nice to standardise the
> naming (such as otc_ vs internode_) as well as the provision of values with
> units - while maintaining perpetual backwards compatibility with the old
> parameter names, of course.
> For temporal units, I would propose parsing strings with suffixes of:
> {{code}}
> u|micros(econds?)?
> ms|millis(econds?)?
> s(econds?)?
> m(inutes?)?
> h(ours?)?
> d(ays?)?
> mo(nths?)?
> {{code}}
> For rate units, I would propose parsing any of the standard {{B/s, KiB/s,
> MiB/s, GiB/s, TiB/s}}.
> Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or
> powers of 1000 such as {{KB/s}}, given these are regularly used for either
> their old or new definition e.g. {{KiB/s}}, or we could support them and
> simply log the value in bytes/s.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 05/13: Extend DurationSpec and DataStorageSpec for smallest unit and transfer denylist parameters to the new framework patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capw
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b9e2ab75f8f6dedd45c6ad7a83b3160149869262 Author: Ekaterina Dimitrova AuthorDate: Wed Feb 2 12:47:41 2022 -0500 Extend DurationSpec and DataStorageSpec for smallest unit and transfer denylist parameters to the new framework patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 --- conf/cassandra.yaml| 14 ++-- src/java/org/apache/cassandra/config/Config.java | 12 ++-- .../org/apache/cassandra/config/Converters.java| 28 .../apache/cassandra/config/DataStorageSpec.java | 49 -- .../cassandra/config/DatabaseDescriptor.java | 46 ++--- .../org/apache/cassandra/config/DurationSpec.java | 75 +- .../config/SmallestDataStorageKibibytes.java | 55 .../config/SmallestDataStorageMebibytes.java | 55 .../config/SmallestDurationMilliseconds.java | 57 .../cassandra/config/SmallestDurationMinutes.java | 57 .../cassandra/config/SmallestDurationSeconds.java | 57 .../apache/cassandra/schema/PartitionDenylist.java | 6 +- .../org/apache/cassandra/service/StorageProxy.java | 16 ++--- .../distributed/test/PartitionDenylistTest.java| 8 +-- .../config/DatabaseDescriptorRefTest.java | 7 +- .../cassandra/config/DatabaseDescriptorTest.java | 4 +- .../LoadOldYAMLBackwardCompatibilityTest.java | 6 +- .../config/SmallestDataStorageKibibytesTest.java | 42 .../config/SmallestDataStorageMebibytesTest.java | 42 .../config/SmallestDurationMillisecondsTest.java | 48 ++ .../config/SmallestDurationMinutesTest.java| 52 +++ .../config/SmallestDurationSecondsTest.java| 50 +++ .../cassandra/service/PartitionDenylistTest.java | 8 +-- 23 files changed, 701 insertions(+), 93 deletions(-) diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index 265bf38..1d7998a 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -841,7 +841,7 @@ rpc_keepalive: true # Uncomment to set socket buffer size for internode communication # Note that when setting this, the buffer size is limited by net.core.wmem_max # and when not setting it it is defined by net.ipv4.tcp_wmem -# internode_socket_receive_buffer_size_in_bytes: +# internode_socket_receive_buffer_size: # Set to true to have Cassandra create a hard link to each sstable # flushed or streamed locally in a backups/ subdirectory of the @@ -1076,19 +1076,19 @@ slow_query_log_timeout_in_ms: 500 # Allows denying configurable access (rw/rr) to operations on configured ks, table, and partitions, intended for use by # operators to manage cluster health vs application access. See CASSANDRA-12106 and CEP-13 for more details. -# enable_partition_denylist: false +# partition_denylist_enabled: false -# enable_denylist_writes: true -# enable_denylist_reads: true -# enable_denylist_range_reads: true +# denylist_writes_enabled: true +# denylist_reads_enabled: true +# denylist_range_reads_enabled: true # The interval at which keys in the cache for denylisting will "expire" and async refresh from the backing DB. # Note: this serves only as a fail-safe, as the usage pattern is expected to be "mutate state, refresh cache" on any # changes to the underlying denylist entries. See documentation for details. -# denylist_refresh_seconds: 600 +# denylist_refresh: 600s # In the event of errors on attempting to load the denylist cache, retry on this interval. -# denylist_initial_load_retry_seconds: 5 +# denylist_initial_load_retry: 5s # We cap the number of denylisted keys allowed per table to keep things from growing unbounded. Nodes will warn above # this limit while allowing new denylisted keys to be inserted. Denied keys are loaded in natural query / clustering diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index 0485cc1..eb67be1 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -546,17 +546,17 @@ public class Config /** This feature allows denying access to operations on certain key partitions, intended for use by operators to * provide another tool to manage cluster health vs application access. See CASSANDRA-12106 and CEP-13 for more details. */ -public volatile boolean enable_partition_denylist = false; +public volatile boolean partition_denylist_enabled = false; -public volatile boolean enable_denylist_writes = true; +public volatile boolean denylist_writes_enabled = true; -public volatile boolean enable_denylist_reads
[cassandra] 10/13: Transfer parameters to the newly introduced configuration framework (6) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Ler
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 23138252f20891c26a3692664c6affaf99e86541
Author: Ekaterina Dimitrova
AuthorDate: Thu Feb 3 23:49:50 2022 -0500
Transfer parameters to the newly introduced configuration framework (6)
patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell,
Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234
---
conf/cassandra.yaml| 40 ++---
doc/native_protocol_v4.spec| 4 +-
doc/native_protocol_v5.spec| 4 +-
.../apache/cassandra/batchlog/BatchlogManager.java | 10 +-
src/java/org/apache/cassandra/config/Config.java | 50 +--
.../cassandra/config/DatabaseDescriptor.java | 166 +++--
.../commitlog/AbstractCommitLogSegmentManager.java | 2 +-
.../db/commitlog/CommitLogSegmentManagerCDC.java | 2 +-
.../db/commitlog/GroupCommitLogService.java| 2 +-
.../cassandra/hints/HintsDispatchExecutor.java | 2 +-
.../apache/cassandra/hints/HintsWriteExecutor.java | 4 +-
.../org/apache/cassandra/io/util/FileUtils.java| 26 ++--
.../apache/cassandra/service/StorageService.java | 8 +-
.../cassandra/service/StorageServiceMBean.java | 6 +-
.../nodetool/SetHintedHandoffThrottleInKB.java | 4 +-
test/conf/cassandra-murmur.yaml| 2 +-
test/conf/cassandra-seeds.yaml | 2 +-
...dra-sslcontextfactory-invalidconfiguration.yaml | 2 +-
test/conf/cassandra-sslcontextfactory.yaml | 2 +-
test/conf/cassandra.yaml | 6 +-
test/conf/unit-test-conf/test-native-port.yaml | 2 +-
.../test/HintedHandoffNodetoolTest.java| 4 +-
.../distributed/test/LargeColumnTest.java | 2 +-
.../cassandra/config/DatabaseDescriptorTest.java | 4 +-
.../commitlog/CommitLogInitWithExceptionTest.java | 1 +
.../commitlog/CommitLogSegmentManagerCDCTest.java | 8 +-
26 files changed, 193 insertions(+), 172 deletions(-)
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 0c4acb8..9f7e657 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -68,7 +68,7 @@ max_hint_window: 3h
# are two nodes in the cluster, each delivery thread will use the maximum
# rate; if there are three, each will throttle to half of the maximum,
# since we expect two nodes to be delivering hints simultaneously.)
-hinted_handoff_throttle_in_kb: 1024
+hinted_handoff_throttle: 1024KiB
# Number of threads with which to deliver hints;
# Consider increasing this number when you have multi-dc deployments, since
@@ -81,10 +81,10 @@ max_hints_delivery_threads: 2
# How often hints should be flushed from the internal buffers to disk.
# Will *not* trigger fsync.
-hints_flush_period_in_ms: 1
+hints_flush_period: 1ms
# Maximum size for a single hints file, in megabytes.
-max_hints_file_size_in_mb: 128
+max_hints_file_size: 128MiB
# Enable / disable automatic cleanup for the expired and orphaned hints file.
# Disable the option in order to preserve those hints on the disk.
@@ -114,7 +114,7 @@ auto_hints_cleanup_enabled: false
# Maximum throttle in KBs per second, total. This will be
# reduced proportionally to the number of nodes in the cluster.
-batchlog_replay_throttle_in_kb: 1024
+batchlog_replay_throttle: 1024KiB
# Authentication backend, implementing IAuthenticator; used to identify users
# Out of the box, Cassandra provides
org.apache.cassandra.auth.{AllowAllAuthenticator,
@@ -456,19 +456,19 @@ counter_cache_save_period: 7200
#
# group mode is similar to batch mode, where Cassandra will not ack writes
# until the commit log has been flushed to disk. The difference is group
-# mode will wait up to commitlog_sync_group_window_in_ms between flushes.
+# mode will wait up to commitlog_sync_group_window between flushes.
#
-# commitlog_sync_group_window_in_ms: 1000
+# commitlog_sync_group_window: 1000ms
#
# the default option is "periodic" where writes may be acked immediately
-# and the CommitLog is simply synced every commitlog_sync_period_in_ms
+# and the CommitLog is simply synced every commitlog_sync_period
# milliseconds.
commitlog_sync: periodic
-commitlog_sync_period_in_ms: 1
+commitlog_sync_period: 1ms
# When in periodic commitlog mode, the number of milliseconds to block writes
# while waiting for a slow disk flush to complete.
-# periodic_commitlog_sync_lag_block_in_ms:
+# periodic_commitlog_sync_lag_block:
# The size of the individual commitlog file segments. A commitlog
# segment may be archived, deleted, or recycled once all the data
@@ -479,14 +479,14 @@ commitlog_sync_period_in_ms: 1
# archiving commitlog segments (see commitlog_archiving.properties),
# then you probably want a finer granularity of archiving; 8 or 16 MB
# is
[cassandra] 01/13: Add new custom types and unit tests for configuration patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-1
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit db9f7a67ec4b03413c10034956e2cf18739ca4b1
Author: Ekaterina Dimitrova
AuthorDate: Tue Dec 14 23:00:56 2021 -0500
Add new custom types and unit tests for configuration
patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell,
Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234
---
CHANGES.txt| 1 +
NEWS.txt | 6 +
.../org/apache/cassandra/config/DataRateSpec.java | 378
.../apache/cassandra/config/DataStorageSpec.java | 397 +
.../org/apache/cassandra/config/DurationSpec.java | 342 ++
.../apache/cassandra/config/DataRateSpecTest.java | 136 +++
.../cassandra/config/DataStorageSpecTest.java | 141
.../apache/cassandra/config/DurationSpecTest.java | 160 +
8 files changed, 1561 insertions(+)
diff --git a/CHANGES.txt b/CHANGES.txt
index 4304aa7..66aae18 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.1
+ * Standardize storage configuration parameters' names. Support unit suffixes.
(CASSANDRA-15234)
* Remove support for Windows (CASSANDRA-16956)
* Runtime-configurable YAML option to prohibit USE statements
(CASSANDRA-17318)
* When streaming sees a ClosedChannelException this triggers the disk failure
policy (CASSANDRA-17116)
diff --git a/NEWS.txt b/NEWS.txt
index ff166df..966a729 100644
--- a/NEWS.txt
+++ b/NEWS.txt
@@ -81,6 +81,12 @@ New features
Upgrading
-
+- There is a new cassandra.yaml version 2. Units suffixes should be
provided for all rates(B/s|MiB/s|KiB/s|MiB/s),
+ memory (KiB|MiB|GiB|B) and duration(d|h|s|ms|us|µs|ns|m)
+ parameters. (CASSANDRA-15234)
+ Backward compatibility with the old cassandra.yaml file will be in place
until at least the next major version.
+- Many cassandra.yaml parameters' names have been changed. Full list can
be found on .. (ADD LINK LATER WHEN PAGE
+ IS CREATED) (CASSANDRA-15234)
- Before you upgrade, if you are using
`cassandra.auth_bcrypt_gensalt_log2_rounds` property,
confirm it is set to value lower than 31 otherwise Cassandra will fail
to start. See CASSANDRA-9384
for further details. You also need to regenerate passwords for users for
who the password
diff --git a/src/java/org/apache/cassandra/config/DataRateSpec.java
b/src/java/org/apache/cassandra/config/DataRateSpec.java
new file mode 100644
index 000..3512513
--- /dev/null
+++ b/src/java/org/apache/cassandra/config/DataRateSpec.java
@@ -0,0 +1,378 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.cassandra.config;
+
+import java.util.Arrays;
+import java.util.Objects;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+
+import com.google.common.primitives.Ints;
+
+import org.apache.cassandra.exceptions.ConfigurationException;
+
+/**
+ * Represents a data rate type used for cassandra configuration. It supports
the opportunity for the users to be able to
+ * add units to the confiuration parameter value. (CASSANDRA-15234)
+ */
+public final class DataRateSpec
+{
+/**
+ * The Regexp used to parse the rate provided as String in cassandra.yaml.
+ */
+private static final Pattern BIT_RATE_UNITS_PATTERN =
Pattern.compile("^(\\d+)(MiB/s|KiB/s|B/s)$");
+
+private final double quantity;
+
+private final DataRateUnit unit;
+
+public DataRateSpec(String value)
+{
+//parse the string field value
+Matcher matcher = BIT_RATE_UNITS_PATTERN.matcher(value);
+
+if (!matcher.find())
+throw new ConfigurationException("Invalid bit rate: " + value + "
Accepted units: MiB/s, KiB/s, B/s where " +
+ "case matters and " + "only
non-negative values are valid");
+
+quantity = Long.parseLong(matcher.group(1));
+unit = DataRateUnit.fromSymbol(matcher.group(2));
+}
+
+DataRateSpec(double
[cassandra] 06/13: Transfer parameters to the newly introduced configuration framework (2) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Ler
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit d85f7f7c2dd4b9bbdb44bc96235e6a8bc3ff3967
Author: Ekaterina Dimitrova
AuthorDate: Thu Feb 3 00:19:28 2022 -0500
Transfer parameters to the newly introduced configuration framework (2)
patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell,
Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234
---
conf/cassandra.yaml | 4 ++--
src/java/org/apache/cassandra/config/Config.java| 4 ++--
src/java/org/apache/cassandra/config/DatabaseDescriptor.java| 6 +++---
.../org/apache/cassandra/config/SmallestDataStorageKibibytes.java | 2 +-
.../org/apache/cassandra/config/SmallestDataStorageMebibytes.java | 2 +-
.../org/apache/cassandra/config/SmallestDurationMilliseconds.java | 2 +-
src/java/org/apache/cassandra/config/SmallestDurationMinutes.java | 2 +-
src/java/org/apache/cassandra/config/SmallestDurationSeconds.java | 2 +-
src/java/org/apache/cassandra/io/sstable/SSTableRewriter.java | 2 +-
.../cassandra/config/LoadOldYAMLBackwardCompatibilityTest.java | 4 ++--
10 files changed, 15 insertions(+), 15 deletions(-)
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 1d7998a..884a432 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -697,7 +697,7 @@ index_summary_resize_interval_in_minutes: 60
# impacting read latencies. Almost always a good idea on SSDs; not
# necessarily on platters.
trickle_fsync: false
-trickle_fsync_interval_in_kb: 10240
+trickle_fsync_interval: 10240KiB
# TCP port, for commands and data
# For security reasons, you should not expose this port to the internet.
Firewall it if needed.
@@ -930,7 +930,7 @@ compaction_throughput: 64MiB/s
# are completely written, and used in place of the prior sstables for
# any range that has been written. This helps to smoothly transfer reads
# between the sstables, reducing page cache churn and keeping hot rows hot
-sstable_preemptive_open_interval_in_mb: 50
+sstable_preemptive_open_interval: 50MiB
# When enabled, permits Cassandra to zero-copy stream entire eligible
# SSTables between nodes, including every component.
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index eb67be1..7602118 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -318,9 +318,9 @@ public class Config
public volatile boolean incremental_backups = false;
public boolean trickle_fsync = false;
-public int trickle_fsync_interval_in_kb = 10240;
+public SmallestDataStorageKibibytes trickle_fsync_interval = new
SmallestDataStorageKibibytes("10240KiB");
-public volatile int sstable_preemptive_open_interval_in_mb = 50;
+public volatile SmallestDataStorageMebibytes
sstable_preemptive_open_interval = new SmallestDataStorageMebibytes("50MiB");
public volatile boolean key_cache_migrate_during_compaction = true;
public Long key_cache_size_in_mb = null;
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index 102a587..297e619 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -2824,11 +2824,11 @@ public class DatabaseDescriptor
public static int getSSTablePreemptiveOpenIntervalInMB()
{
-return conf.sstable_preemptive_open_interval_in_mb;
+return conf.sstable_preemptive_open_interval.toMebibytesAsInt();
}
public static void setSSTablePreemptiveOpenIntervalInMB(int mb)
{
-conf.sstable_preemptive_open_interval_in_mb = mb;
+conf.sstable_preemptive_open_interval =
SmallestDataStorageMebibytes.inMebibytes(mb);
}
public static boolean getTrickleFsync()
@@ -2838,7 +2838,7 @@ public class DatabaseDescriptor
public static int getTrickleFsyncIntervalInKb()
{
-return conf.trickle_fsync_interval_in_kb;
+return conf.trickle_fsync_interval.toKibibytesAsInt();
}
public static long getKeyCacheSizeInMB()
diff --git
a/src/java/org/apache/cassandra/config/SmallestDataStorageKibibytes.java
b/src/java/org/apache/cassandra/config/SmallestDataStorageKibibytes.java
index e298ba5..375af29 100644
--- a/src/java/org/apache/cassandra/config/SmallestDataStorageKibibytes.java
+++ b/src/java/org/apache/cassandra/config/SmallestDataStorageKibibytes.java
@@ -23,7 +23,7 @@ package org.apache.cassandra.config;
* not to lose precision while converting to smaller units (until we migrate
those parameters to use internally the smallest
* supported unit) we restrict those parameters to use only kibibytes or
larger units.
[cassandra] 11/13: Transfer parameters to the newly introduced configuration framework (7) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Ler
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 6d5203615f7a9670cb1698b74123666bc25ba471
Author: Ekaterina Dimitrova
AuthorDate: Fri Feb 4 00:25:14 2022 -0500
Transfer parameters to the newly introduced configuration framework (7)
patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell,
Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234
---
conf/cassandra.yaml| 42
.../org/apache/cassandra/cache/OHCProvider.java| 2 +-
.../cassandra/cache/SerializingCacheProvider.java | 2 +-
src/java/org/apache/cassandra/config/Config.java | 61 +++
.../org/apache/cassandra/config/Converters.java| 13 ++-
.../cassandra/config/DatabaseDescriptor.java | 119 ++---
.../cassandra/config/SmallestDurationSeconds.java | 29 +
.../apache/cassandra/db/virtual/SettingsTable.java | 15 ++-
.../org/apache/cassandra/service/CacheService.java | 8 +-
test/conf/cassandra-murmur.yaml| 2 +-
test/conf/cassandra-old.yaml | 2 +-
test/conf/cassandra-seeds.yaml | 2 +-
...dra-sslcontextfactory-invalidconfiguration.yaml | 4 +-
test/conf/cassandra-sslcontextfactory.yaml | 4 +-
test/conf/cassandra.yaml | 4 +-
test/conf/unit-test-conf/test-native-port.yaml | 4 +-
.../cassandra/distributed/impl/InstanceConfig.java | 8 +-
.../cassandra/distributed/test/NodeToolTest.java | 2 +-
.../cassandra/simulator/ClusterSimulation.java | 4 +-
.../LoadOldYAMLBackwardCompatibilityTest.java | 21 ++--
test/unit/org/apache/cassandra/cql3/CQLTester.java | 7 +-
.../cql3/validation/entities/JsonTest.java | 4 +-
22 files changed, 214 insertions(+), 145 deletions(-)
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 9f7e657..cf4326a 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -344,8 +344,8 @@ commit_failure_policy: stop
# fit in the cache. In most cases it is not neccessary to change this value.
# Constantly re-preparing statements is a performance penalty.
#
-# Default value ("auto") is 1/256th of the heap or 10MB, whichever is greater
-prepared_statements_cache_size_mb:
+# Default value ("auto") is 1/256th of the heap or 10MiB, whichever is greater
+# prepared_statements_cache_size:
# Maximum size of the key cache in memory.
#
@@ -359,7 +359,7 @@ prepared_statements_cache_size_mb:
# NOTE: if you reduce the size, you may not get you hottest keys loaded on
startup.
#
# Default value is empty to make it "auto" (min(5% of Heap (in MB), 100MB)).
Set to 0 to disable key cache.
-key_cache_size_in_mb:
+# key_cache_size:
# Duration in seconds after which Cassandra should
# save the key cache. Caches are saved to saved_caches_directory as
@@ -370,7 +370,7 @@ key_cache_size_in_mb:
# has limited use.
#
# Default is 14400 or 4 hours.
-key_cache_save_period: 14400
+key_cache_save_period: 4h
# Number of keys from the key cache to save
# Disabled by default, meaning all keys are going to be saved
@@ -394,7 +394,7 @@ key_cache_save_period: 14400
# headroom for OS block level cache. Do never allow your system to swap.
#
# Default value is 0, to disable row caching.
-row_cache_size_in_mb: 0
+row_cache_size: 0MiB
# Duration in seconds after which Cassandra should save the row cache.
# Caches are saved to saved_caches_directory as specified in this
configuration file.
@@ -404,7 +404,7 @@ row_cache_size_in_mb: 0
# has limited use.
#
# Default is 0 to disable saving the row cache.
-row_cache_save_period: 0
+row_cache_save_period: 0s
# Number of keys from the row cache to save.
# Specify 0 (which is the default), meaning all keys are going to be saved
@@ -423,14 +423,14 @@ row_cache_save_period: 0
#
# Default value is empty to make it "auto" (min(2.5% of Heap (in MB), 50MB)).
Set to 0 to disable counter cache.
# NOTE: if you perform counter deletes and rely on low gcgs, you should
disable the counter cache.
-counter_cache_size_in_mb:
+# counter_cache_size:
# Duration in seconds after which Cassandra should
# save the counter cache (keys only). Caches are saved to
saved_caches_directory as
# specified in this configuration file.
#
# Default is 7200 or 2 hours.
-counter_cache_save_period: 7200
+counter_cache_save_period: 7200s
# Number of keys from the counter cache to save
# Disabled by default, meaning all keys are going to be saved
@@ -443,7 +443,7 @@ counter_cache_save_period: 7200
# Number of seconds the server will wait for each cache (row, key, etc ...) to
load while starting
# the Cassandra process. Setting this to a negative value is equivalent to
disabling all cache loading on startup
# while still having the cache during runtime.
-# cache_load_timeout_seconds: 30
+#
[cassandra] 08/13: Transfer parameters to the newly introduced configuration framework (4) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Ler
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit ed48f3c017c5e572a523890bcd5b7c798d7eb358 Author: Ekaterina Dimitrova AuthorDate: Thu Feb 3 16:43:36 2022 -0500 Transfer parameters to the newly introduced configuration framework (4) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 --- conf/cassandra.yaml| 18 ++--- src/java/org/apache/cassandra/config/Config.java | 30 --- .../cassandra/config/DatabaseDescriptor.java | 94 -- src/java/org/apache/cassandra/db/Keyspace.java | 4 +- .../apache/cassandra/streaming/StreamSession.java | 2 +- .../cassandra/distributed/test/CASAddTest.java | 4 +- .../apache/cassandra/distributed/test/CASTest.java | 53 ++-- .../distributed/test/LargeColumnTest.java | 4 +- .../distributed/test/MessageFiltersTest.java | 3 +- .../test/ReadRepairEmptyRangeTombstonesTest.java | 5 +- .../distributed/test/ReadRepairQueryTypesTest.java | 4 +- .../cassandra/distributed/test/ReadRepairTest.java | 2 +- .../test/ring/ReadsDuringBootstrapTest.java| 4 +- .../upgrade/MixedModeAvailabilityTestBase.java | 5 +- .../upgrade/MixedModeConsistencyTestBase.java | 5 +- .../upgrade/MixedModeMessageForwardTest.java | 1 - .../cassandra/simulator/ClusterSimulation.java | 8 +- .../cassandra/config/DatabaseDescriptorTest.java | 65 --- .../LoadOldYAMLBackwardCompatibilityTest.java | 4 +- 19 files changed, 146 insertions(+), 169 deletions(-) diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index e9cce68..3d8168b 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -984,28 +984,28 @@ sstable_preemptive_open_interval: 50MiB # How long the coordinator should wait for read operations to complete. # Lowest acceptable value is 10 ms. -read_request_timeout_in_ms: 5000 +read_request_timeout: 5000ms # How long the coordinator should wait for seq or index scans to complete. # Lowest acceptable value is 10 ms. -range_request_timeout_in_ms: 1 +range_request_timeout: 1ms # How long the coordinator should wait for writes to complete. # Lowest acceptable value is 10 ms. -write_request_timeout_in_ms: 2000 +write_request_timeout: 2000ms # How long the coordinator should wait for counter writes to complete. # Lowest acceptable value is 10 ms. -counter_write_request_timeout_in_ms: 5000 +counter_write_request_timeout: 5000ms # How long a coordinator should continue to retry a CAS operation # that contends with other proposals for the same row. # Lowest acceptable value is 10 ms. -cas_contention_timeout_in_ms: 1000 +cas_contention_timeout: 1000ms # How long the coordinator should wait for truncates to complete # (This can be much longer, because unless auto_snapshot is disabled # we need to flush first so we can snapshot before removing the data.) # Lowest acceptable value is 10 ms. -truncate_request_timeout_in_ms: 6 +truncate_request_timeout: 6ms # The default timeout for other, miscellaneous operations. # Lowest acceptable value is 10 ms. -request_timeout_in_ms: 1 +request_timeout: 1ms # Defensive settings for protecting Cassandra from true network partitions. # See (CASSANDRA-14358) for details. @@ -1049,7 +1049,7 @@ request_timeout_in_ms: 1 # How long before a node logs slow queries. Select queries that take longer than # this timeout to execute, will generate an aggregated log message, so that slow queries # can be identified. Set this value to zero to disable slow query logging. -slow_query_log_timeout_in_ms: 500 +slow_query_log_timeout: 500ms # Enable operation timeout information exchange between nodes to accurately # measure request timeouts. If disabled, replicas will assume that requests @@ -1067,7 +1067,7 @@ slow_query_log_timeout_in_ms: 500 # 2 keep-alive cycles the stream session times out and fail # Default value is 300s (5 minutes), which means stalled stream # times out in 10 minutes by default -# streaming_keep_alive_period_in_secs: 300 +# streaming_keep_alive_period: 300s # Limit number of connections per host for streaming # Increase this when you notice that joins are CPU-bound rather that network diff --git a/src/java/org/apache/cassandra/config/Config.java b/src/java/org/apache/cassandra/config/Config.java index 172f16b..6c22d5f 100644 --- a/src/java/org/apache/cassandra/config/Config.java +++ b/src/java/org/apache/cassandra/config/Config.java @@ -102,29 +102,39 @@ public class Config /** Triggers automatic allocation of tokens if set, based on the provided replica count for a datacenter */ public Integer allocate_tokens_for_local_replication_factor = null; -public long native_transport_idle_timeout_in_ms = 0L; +
[cassandra] 13/13: Remove old Duration class in favor of DurationSpec class patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDR
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 9f56bf4ca7fdb61ad09e5f2ad09b87cd01e0716b
Author: Ekaterina Dimitrova
AuthorDate: Sat Feb 5 17:51:32 2022 -0500
Remove old Duration class in favor of DurationSpec class
patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell,
Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234
---
conf/cassandra.yaml| 20 +-
src/java/org/apache/cassandra/config/Duration.java | 276 -
.../org/apache/cassandra/db/ColumnFamilyStore.java | 9 +-
src/java/org/apache/cassandra/db/Keyspace.java | 4 +-
.../apache/cassandra/service/StorageService.java | 8 +-
.../service/snapshot/SnapshotManifest.java | 4 +-
.../apache/cassandra/tools/nodetool/Snapshot.java | 4 +-
.../org/apache/cassandra/config/DurationTest.java | 60 -
.../org/apache/cassandra/db/DirectoriesTest.java | 7 +-
.../service/snapshot/SnapshotManifestTest.java | 4 +-
10 files changed, 30 insertions(+), 366 deletions(-)
diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml
index 7e39097..71fb562 100644
--- a/conf/cassandra.yaml
+++ b/conf/cassandra.yaml
@@ -441,7 +441,7 @@ counter_cache_save_period: 7200s
# saved_caches_directory: /var/lib/cassandra/saved_caches
# Number of seconds the server will wait for each cache (row, key, etc ...) to
load while starting
-# the Cassandra process. Setting this to a negative value is equivalent to
disabling all cache loading on startup
+# the Cassandra process. Setting this to zero is equivalent to disabling all
cache loading on startup
# while still having the cache during runtime.
# cache_load_timeout: 30s
@@ -1467,15 +1467,15 @@ audit_logging_options:
# default options for full query logging - these can be overridden from
command line when executing
# nodetool enablefullquerylog
-#full_query_logging_options:
-# log_dir:
-# roll_cycle: HOURLY
-# block: true
-# max_queue_weight: 268435456 # 256 MiB
-# max_log_size: 17179869184 # 16 GiB
-## archive command is "/path/to/script.sh %path" where %path is replaced
with the file being rolled:
-# archive_command:
-# max_archive_retries: 10
+# full_query_logging_options:
+ # log_dir:
+ # roll_cycle: HOURLY
+ # block: true
+ # max_queue_weight: 268435456 # 256 MiB
+ # max_log_size: 17179869184 # 16 GiB
+ ## archive command is "/path/to/script.sh %path" where %path is replaced
with the file being rolled:
+ # archive_command:
+ # max_archive_retries: 10
# validate tombstones on reads and compaction
# can be either "disabled", "warn" or "exception"
diff --git a/src/java/org/apache/cassandra/config/Duration.java
b/src/java/org/apache/cassandra/config/Duration.java
deleted file mode 100644
index 89de354..000
--- a/src/java/org/apache/cassandra/config/Duration.java
+++ /dev/null
@@ -1,276 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one
- * or more contributor license agreements. See the NOTICE file
- * distributed with this work for additional information
- * regarding copyright ownership. The ASF licenses this file
- * to you under the Apache License, Version 2.0 (the
- * "License"); you may not use this file except in compliance
- * with the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-package org.apache.cassandra.config;
-
-import java.util.Arrays;
-import java.util.Objects;
-import java.util.concurrent.TimeUnit;
-import java.util.regex.Matcher;
-import java.util.regex.Pattern;
-import java.util.stream.Collectors;
-
-import com.google.common.primitives.Ints;
-
-/**
- * Represents a positive time duration.
- */
-public final class Duration
-{
-/**
- * The Regexp used to parse the duration provided as String.
- */
-private static final Pattern TIME_UNITS_PATTERN =
Pattern.compile(("^(\\d+)([a-zA-Z]{1,2}|µs|µS)$"));
-private static final Pattern DOUBLE_TIME_UNITS_PATTERN =
Pattern.compile(("^(\\d+\\.\\d+)([a-zA-Z]{1,2}|µs|µS)$"));
-
-private final long quantity;
-
-private final TimeUnit unit;
-
-
-public Duration(String value)
-{
-if (value == null || value.equals("null"))
-{
-quantity = 0;
-unit = TimeUnit.MILLISECONDS;
-return;
-}
-
-//parse the string field value
-Matcher matcher = TIME_UNITS_PATTERN.matcher(value);
-Matcher matcherDouble =
[cassandra] 09/13: Transfer parameters to the newly introduced configuration framework (5) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Ler
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 1315d0c96f4625a76296f58d431f97669e5178c2 Author: Ekaterina Dimitrova AuthorDate: Thu Feb 3 22:28:41 2022 -0500 Transfer parameters to the newly introduced configuration framework (5) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 --- conf/cassandra.yaml| 40 +-- src/java/org/apache/cassandra/config/Config.java | 79 +++--- .../cassandra/config/DatabaseDescriptor.java | 272 - .../config/SmallestDataStorageMebibytes.java | 11 + .../cassandra/cql3/statements/BatchStatement.java | 4 +- src/java/org/apache/cassandra/db/ColumnIndex.java | 12 +- src/java/org/apache/cassandra/db/Memtable.java | 4 +- .../org/apache/cassandra/db/RowIndexEntry.java | 16 +- .../apache/cassandra/db/marshal/AbstractType.java | 2 +- .../org/apache/cassandra/io/sstable/IndexInfo.java | 2 +- .../org/apache/cassandra/io/util/FileUtils.java| 10 +- .../apache/cassandra/repair/ValidationManager.java | 2 +- .../cassandra/service/ActiveRepairService.java | 20 +- .../service/ActiveRepairServiceMBean.java | 5 + .../apache/cassandra/service/StorageService.java | 18 +- .../apache/cassandra/streaming/StreamSession.java | 2 +- .../tools/nodetool/GetColumnIndexSize.java | 2 +- .../tools/nodetool/SetColumnIndexSize.java | 6 +- test/conf/cassandra-murmur.yaml| 2 +- test/conf/cassandra-old.yaml | 3 +- ...dra-sslcontextfactory-invalidconfiguration.yaml | 2 +- test/conf/cassandra-sslcontextfactory.yaml | 2 +- test/conf/unit-test-conf/test-native-port.yaml | 2 +- .../cassandra/distributed/impl/InstanceConfig.java | 2 +- .../distributed/test/LargeColumnTest.java | 8 +- .../cassandra/simulator/ClusterSimulation.java | 2 +- .../cassandra/config/DatabaseDescriptorTest.java | 39 +-- .../LoadOldYAMLBackwardCompatibilityTest.java | 13 +- .../config/YamlConfigurationLoaderTest.java| 8 +- .../db/compaction/CompactionsCQLTest.java | 6 +- .../io/sstable/SSTableWriterTestBase.java | 2 +- .../org/apache/cassandra/repair/ValidatorTest.java | 16 +- .../cassandra/service/ClientWarningsTest.java | 2 +- .../cassandra/service/ProtocolBetaVersionTest.java | 2 +- .../tools/nodetool/SetGetColumnIndexSizeTest.java | 12 +- .../cassandra/transport/CQLConnectionTest.java | 4 +- 36 files changed, 365 insertions(+), 269 deletions(-) diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index 3d8168b..0c4acb8 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -582,8 +582,8 @@ concurrent_materialized_view_writes: 32 # accepting writes when the limit is exceeded until a flush completes, # and will trigger a flush based on memtable_cleanup_threshold # If omitted, Cassandra will set both to 1/4 the size of the heap. -# memtable_heap_space_in_mb: 2048 -# memtable_offheap_space_in_mb: 2048 +# memtable_heap_space: 2048MiB +# memtable_offheap_space: 2048MiB # memtable_cleanup_threshold is deprecated. The default calculation # is the only reasonable choice. See the comments on memtable_flush_writers @@ -620,7 +620,7 @@ memtable_allocation_type: heap_buffers # # For more details see https://issues.apache.org/jira/browse/CASSANDRA-14096. # -# repair_session_space_in_mb: +# repair_session_space: # Total space to use for commit logs on disk. # @@ -771,7 +771,7 @@ native_transport_port: 9042 # The maximum size of allowed frame. Frame (requests) larger than this will # be rejected as invalid. The default is 16MB. If you're changing this parameter, # you may want to adjust max_value_size_in_mb accordingly. This should be positive and less than 2048. -# native_transport_max_frame_size_in_mb: 16 +# native_transport_max_frame_size: 16MiB # The maximum number of concurrent client connections. # The default is -1, which means unlimited. @@ -836,7 +836,7 @@ rpc_keepalive: true # /proc/sys/net/ipv4/tcp_wmem # /proc/sys/net/ipv4/tcp_wmem # and 'man tcp' -# internode_socket_send_buffer_size_in_bytes: +# internode_socket_send_buffer_size: # Uncomment to set socket buffer size for internode communication # Note that when setting this, the buffer size is limited by net.core.wmem_max @@ -878,7 +878,7 @@ snapshot_links_per_second: 0 # - but, Cassandra will keep the collation index in memory for hot # rows (as part of the key cache), so a larger granularity means # you can cache more hot rows -column_index_size_in_kb: 64 +column_index_size: 64KiB # Per sstable indexed key cache entries (the collation index in memory # mentioned above) exceeding this size will not be held on
[cassandra] 03/13: DataRate parameters transition to the new framework Fix the DB descriptorRefTest which failed on the previous commit patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 5bb4bab12f8edfef95ed13cbabf8c0f377986065 Author: Ekaterina Dimitrova AuthorDate: Mon Jan 31 21:51:49 2022 -0500 DataRate parameters transition to the new framework Fix the DB descriptorRefTest which failed on the previous commit patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 --- .build/build-rat.xml | 2 +- conf/cassandra.yaml| 24 +++--- src/java/org/apache/cassandra/config/Config.java | 13 +-- .../org/apache/cassandra/config/Converters.java| 4 +- .../org/apache/cassandra/config/DataRateSpec.java | 2 + .../cassandra/config/DatabaseDescriptor.java | 57 ++ .../cassandra/db/compaction/CompactionManager.java | 10 +-- .../apache/cassandra/service/StorageService.java | 61 ++ .../cassandra/service/StorageServiceMBean.java | 17 +++- .../apache/cassandra/streaming/StreamManager.java | 36 - .../streaming/StreamingDataOutputPlus.java | 2 +- .../org/apache/cassandra/tools/LoaderOptions.java | 12 +-- src/java/org/apache/cassandra/tools/NodeProbe.java | 16 ++-- .../tools/nodetool/GetCompactionThroughput.java| 4 +- .../tools/nodetool/GetInterDCStreamThroughput.java | 6 +- .../tools/nodetool/GetStreamThroughput.java| 6 +- .../tools/nodetool/SetCompactionThroughput.java| 4 +- .../tools/nodetool/SetInterDCStreamThroughput.java | 4 +- .../tools/nodetool/SetStreamThroughput.java| 4 +- test/conf/cassandra-murmur.yaml| 2 +- ...ed_parameters_names.yaml => cassandra-old.yaml} | 2 +- test/conf/cassandra-seeds.yaml | 2 +- ...dra-sslcontextfactory-invalidconfiguration.yaml | 4 +- test/conf/cassandra-sslcontextfactory.yaml | 4 +- test/conf/cassandra.yaml | 4 +- test/conf/unit-test-conf/test-native-port.yaml | 2 +- .../test/AbstractNetstatsBootstrapStreaming.java | 8 +- ...WithEntireSSTablesCompressionStreamingTest.java | 2 +- .../test/NetstatsRepairStreamingTest.java | 4 +- .../cassandra/streaming/LongStreamingTest.java | 20 ++--- .../microbench/ZeroCopyStreamingBenchmark.java | 2 +- .../config/DatabaseDescriptorRefTest.java | 14 +++- .../LoadOldYAMLBackwardCompatibilityTest.java | 89 - .../miscellaneous/CrcCheckChanceTest.java | 2 +- .../net/AsyncStreamingOutputPlusTest.java | 8 +- .../cassandra/streaming/StreamManagerTest.java | 92 ++ .../cassandra/streaming/StreamRateLimiterTest.java | 32 ...st.java => SetGetCompactionThroughputTest.java} | 41 -- ...etEntireSSTableInterDCStreamThroughputTest.java | 12 +-- .../SetGetEntireSSTableStreamThroughputTest.java | 12 +-- .../SetGetInterDCStreamThroughputTest.java | 26 +++--- .../tools/nodetool/SetGetStreamThroughputTest.java | 26 +++--- 42 files changed, 422 insertions(+), 272 deletions(-) diff --git a/.build/build-rat.xml b/.build/build-rat.xml index a1a17cd..599d5ea 100644 --- a/.build/build-rat.xml +++ b/.build/build-rat.xml @@ -55,7 +55,7 @@ - + diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index a18a5b0..8741b9b 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -896,7 +896,7 @@ column_index_cache_size_in_kb: 2 # during a single long running compactions. The default is usually # fine and if you experience problems with compaction running too # slowly or too fast, you should look at -# compaction_throughput_mb_per_sec first. +# compaction_throughput first. # # concurrent_compactors defaults to the smaller of (number of disks, # number of cores), with a minimum of 2 and a maximum of 8. @@ -924,7 +924,7 @@ concurrent_materialized_view_builders: 1 # Setting this to 0 disables throttling. Note that this accounts for all types # of compaction, including validation compaction (building Merkle trees # for repairs). -compaction_throughput_mb_per_sec: 64 +compaction_throughput: 64MiB/s # When compacting, the replacement sstable(s) can be opened before they # are completely written, and used in place of the prior sstables for @@ -935,8 +935,8 @@ sstable_preemptive_open_interval_in_mb: 50 # When enabled, permits Cassandra to zero-copy stream entire eligible # SSTables between nodes, including every component. # This speeds up the network transfer significantly subject to -# throttling specified by entire_sstable_stream_throughput_outbound_megabits_per_sec, -# and entire_sstable_inter_dc_stream_throughput_outbound_megabits_per_sec
[cassandra] 07/13: Transfer parameters to the newly introduced configuration framework (3) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Ler
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 755fd9446b084e659e98bd7336b9e910c2e12577 Author: Ekaterina Dimitrova AuthorDate: Thu Feb 3 14:39:48 2022 -0500 Transfer parameters to the newly introduced configuration framework (3) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 --- conf/cassandra.yaml| 36 +-- src/java/org/apache/cassandra/auth/AuthConfig.java | 6 ++-- .../apache/cassandra/auth/AuthenticatedUser.java | 2 +- src/java/org/apache/cassandra/auth/Roles.java | 2 +- src/java/org/apache/cassandra/config/Config.java | 21 .../cassandra/config/DatabaseDescriptor.java | 40 +++--- .../apache/cassandra/service/StorageService.java | 2 +- .../test/BootstrapBinaryDisabledTest.java | 4 +-- .../db/virtual/CredentialsCacheKeysTableTest.java | 2 +- .../virtual/JmxPermissionsCacheKeysTableTest.java | 2 +- .../NetworkPermissionsCacheKeysTableTest.java | 2 +- .../db/virtual/PermissionsCacheKeysTableTest.java | 2 +- .../db/virtual/RolesCacheKeysTableTest.java| 2 +- 13 files changed, 65 insertions(+), 58 deletions(-) diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index 884a432..e9cce68 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -61,7 +61,7 @@ hinted_handoff_enabled: true # this defines the maximum amount of time a dead host will have hints # generated. After it has been dead this long, new hints for it will not be # created until it has been seen alive and gone down again. -max_hint_window_in_ms: 1080 # 3 hours +max_hint_window: 3h # Maximum throttle in KBs per second, per delivery thread. This will be # reduced proportionally to the number of nodes in the cluster. (If there @@ -101,10 +101,10 @@ auto_hints_cleanup_enabled: false # Enable / disable persistent hint windows. # # If set to false, a hint will be stored only in case a respective node -# that hint is for is down less than or equal to max_hint_window_in_ms. +# that hint is for is down less than or equal to max_hint_window. # # If set to true, a hint will be stored in case there is not any -# hint which was stored earlier than max_hint_window_in_ms. This is for cases +# hint which was stored earlier than max_hint_window. This is for cases # when a node keeps to restart and hints are not delivered yet, we would be saving # hints for that node indefinitely. # @@ -172,21 +172,21 @@ network_authorizer: AllowAllNetworkAuthorizer # Will be disabled automatically for AllowAllAuthenticator. # For a long-running cache using roles_cache_active_update, consider # setting to something longer such as a daily validation: 8640 -roles_validity_in_ms: 2000 +roles_validity: 2000ms # Refresh interval for roles cache (if enabled). # After this interval, cache entries become eligible for refresh. Upon next # access, an async reload is scheduled and the old value returned until it -# completes. If roles_validity_in_ms is non-zero, then this must be +# completes. If roles_validity is non-zero, then this must be # also. # This setting is also used to inform the interval of auto-updating if # using roles_cache_active_update. -# Defaults to the same value as roles_validity_in_ms. +# Defaults to the same value as roles_validity. # For a long-running cache, consider setting this to 6 (1 hour) etc. -# roles_update_interval_in_ms: 2000 +# roles_update_interval: 2000ms # If true, cache contents are actively updated by a background task at the -# interval set by roles_update_interval_in_ms. If false, cache entries +# interval set by roles_update_interval. If false, cache entries # become eligible for refresh after their update interval. Upon next access, # an async reload is scheduled and the old value returned until it completes. # roles_cache_active_update: false @@ -197,21 +197,21 @@ roles_validity_in_ms: 2000 # Will be disabled automatically for AllowAllAuthorizer. # For a long-running cache using permissions_cache_active_update, consider # setting to something longer such as a daily validation: 8640 -permissions_validity_in_ms: 2000 +permissions_validity: 2000ms # Refresh interval for permissions cache (if enabled). # After this interval, cache entries become eligible for refresh. Upon next # access, an async reload is scheduled and the old value returned until it -# completes. If permissions_validity_in_ms is non-zero, then this must be +# completes. If permissions_validity is non-zero, then this must be # also. # This setting is also used to inform the interval of auto-updating if # using permissions_cache_active_update. -# Defaults to the same value as permissions_validity_in_ms. +# Defaults to the same value as
[cassandra] 04/13: Transfer parameters to the newly introduced configuration framework (1) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Ler
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit a3258d66bcc9f946304c19d59e75d2721126303e Author: Ekaterina Dimitrova AuthorDate: Tue Feb 1 17:14:17 2022 -0500 Transfer parameters to the newly introduced configuration framework (1) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 --- conf/cassandra.yaml| 20 +-- pylib/cassandra-cqlsh-tests.sh | 4 +-- src/java/org/apache/cassandra/config/Config.java | 33 +++-- .../cassandra/config/DatabaseDescriptor.java | 40 ++--- .../apache/cassandra/config/EncryptionOptions.java | 41 +++--- .../cassandra/cql3/functions/UDFunction.java | 4 +-- .../statements/schema/CreateIndexStatement.java| 2 +- .../statements/schema/CreateViewStatement.java | 2 +- .../apache/cassandra/db/virtual/SettingsTable.java | 2 +- .../org/apache/cassandra/net/InboundSockets.java | 2 +- .../apache/cassandra/net/OutboundConnection.java | 2 +- test/conf/cassandra-murmur.yaml| 8 ++--- test/conf/cassandra-seeds.yaml | 4 +-- ...dra-sslcontextfactory-invalidconfiguration.yaml | 8 ++--- test/conf/cassandra-sslcontextfactory.yaml | 8 ++--- test/conf/cassandra.yaml | 8 ++--- test/conf/unit-test-conf/test-native-port.yaml | 4 +-- .../cassandra/distributed/test/CountersTest.java | 2 +- .../cassandra/distributed/test/GroupByTest.java| 6 ++-- .../test/InternodeEncryptionOptionsTest.java | 6 ++-- .../upgrade/CompactStorageUpgradeTest.java | 2 +- .../LoadOldYAMLBackwardCompatibilityTest.java | 17 + test/unit/org/apache/cassandra/cql3/ViewTest.java | 6 ++-- .../apache/cassandra/index/sasi/SASICQLTest.java | 6 ++-- .../apache/cassandra/net/MessagingServiceTest.java | 4 +-- 25 files changed, 128 insertions(+), 113 deletions(-) diff --git a/conf/cassandra.yaml b/conf/cassandra.yaml index 8741b9b..265bf38 100644 --- a/conf/cassandra.yaml +++ b/conf/cassandra.yaml @@ -1059,7 +1059,7 @@ slow_query_log_timeout_in_ms: 500 # # Warning: It is generally assumed that users have setup NTP on their clusters, and that clocks are modestly in sync, # since this is a requirement for general correctness of last write wins. -#cross_node_timeout: true +# internode_timeout: true # Set keep-alive period for streaming # This node will send a keep-alive message periodically with this period. @@ -1225,7 +1225,7 @@ server_encryption_options: # optional: true # If enabled, will open up an encrypted listening socket on ssl_storage_port. Should only be used # during upgrade to 4.0; otherwise, set to false. -enable_legacy_ssl_storage_port: false +legacy_ssl_storage_port_enabled: false # Set to a valid keystore if internode_encryption is dc, rack or all keystore: conf/.keystore keystore_password: cassandra @@ -1311,13 +1311,13 @@ tracetype_repair_ttl: 604800 # INFO level # UDFs (user defined functions) are disabled by default. # As of Cassandra 3.0 there is a sandbox in place that should prevent execution of evil code. -enable_user_defined_functions: false +user_defined_functions_enabled: false # Enables scripted UDFs (JavaScript UDFs). -# Java UDFs are always enabled, if enable_user_defined_functions is true. +# Java UDFs are always enabled, if user_defined_functions_enabled is true. # Enable this option to be able to use UDFs with "language javascript" or any custom JSR-223 provider. -# This option has no effect, if enable_user_defined_functions is false. -enable_scripted_user_defined_functions: false +# This option has no effect, if user_defined_functions_enabled is false. +scripted_user_defined_functions_enabled: false # Enables encrypting data at-rest (on disk). Different key providers can be plugged in, but the default reads from # a JCE-style keystore. A single keystore can hold multiple keys, but the one referenced by @@ -1528,19 +1528,19 @@ report_unconfirmed_repaired_data_mismatches: false # Enables materialized view creation on this node. # Materialized views are considered experimental and are not recommended for production use. -enable_materialized_views: false +materialized_views_enabled: false # Enables SASI index creation on this node. # SASI indexes are considered experimental and are not recommended for production use. -enable_sasi_indexes: false +sasi_indexes_enabled: false # Enables creation of transiently replicated keyspaces on this node. # Transient replication is experimental and is not recommended for production use. -enable_transient_replication: false +transient_replication_enabled: false # Enables the used of 'ALTER ... DROP
[cassandra] branch trunk updated (da47849 -> 9f56bf4)
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from da47849 Remove Windows-specific classes and related code new db9f7a6 Add new custom types and unit tests for configuration patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new 9c6b382 Backward compatibility framework for configuration parameters patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new 5bb4bab DataRate parameters transition to the new framework Fix the DB descriptorRefTest which failed on the previous commit patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new a3258d6 Transfer parameters to the newly introduced configuration framework (1) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new b9e2ab7 Extend DurationSpec and DataStorageSpec for smallest unit and transfer denylist parameters to the new framework patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new d85f7f7 Transfer parameters to the newly introduced configuration framework (2) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new 755fd94 Transfer parameters to the newly introduced configuration framework (3) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new ed48f3c Transfer parameters to the newly introduced configuration framework (4) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new 1315d0c Transfer parameters to the newly introduced configuration framework (5) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new 2313825 Transfer parameters to the newly introduced configuration framework (6) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new 6d52036 Transfer parameters to the newly introduced configuration framework (7) patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new c51a7c6 Bulk change of units around the code to support the move to the new configuration framework patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 new 9f56bf4 Remove old Duration class in favor of DurationSpec class patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 The 13 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/build-rat.xml | 2 +- CHANGES.txt| 1 + NEWS.txt | 6 + conf/cassandra.yaml| 462 +-- doc/native_protocol_v4.spec| 4 +- doc/native_protocol_v5.spec| 4 +- pylib/cassandra-cqlsh-tests.sh | 4 +- src/java/org/apache/cassandra/auth/AuthConfig.java | 6 +- .../apache/cassandra/auth/AuthenticatedUser.java | 2 +- src/java/org/apache/cassandra/auth/Roles.java | 2 +- .../apache/cassandra/batchlog/BatchlogManager.java | 10 +- .../apache/cassandra/cache/AutoSavingCache.java| 2 +- .../org/apache/cassandra/cache/CaffeineCache.java | 2 +- .../org/apache/cassandra/cache/ChunkCache.java | 5 +- .../org/apache/cassandra/cache/OHCProvider.java| 2 +- .../apache/cassandra/cache/SerializingCache.java | 2 +- .../cassandra/cache/SerializingCacheProvider.java | 2 +- src/java/org/apache/cassandra/config/Config.java | 315 +--- .../org/apache/cassandra/config/Converters.java| 138 .../org/apache/cassandra/config/DataRateSpec.java | 378 + .../apache/cassandra/config/DataStorageSpec.java | 438 ++ .../cassandra/config/DatabaseDescriptor.java | 888 +++-- src/java/org/apache/cassandra/config/Duration.java | 276 --- .../org/apache/cassandra/config/DurationSpec.java | 387
[cassandra] 02/13: Backward compatibility framework for configuration parameters patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CAS
This is an automated email from the ASF dual-hosted git repository.
edimitrova pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 9c6b382058578ac75b88055a13aa83944901fb88
Author: Ekaterina Dimitrova
AuthorDate: Tue Dec 14 23:04:43 2021 -0500
Backward compatibility framework for configuration parameters
patch by Ekaterina Dimitrova; reviewed by Caleb Rackliffe, David Capwell,
Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234
---
.../org/apache/cassandra/config/Converters.java| 125 +++
src/java/org/apache/cassandra/config/Replaces.java | 9 +-
.../org/apache/cassandra/config/ReplacesList.java | 2 +-
.../cassandra/config/YamlConfigurationLoader.java | 134 ++---
.../apache/cassandra/db/virtual/SettingsTable.java | 16 +++
5 files changed, 263 insertions(+), 23 deletions(-)
diff --git a/src/java/org/apache/cassandra/config/Converters.java
b/src/java/org/apache/cassandra/config/Converters.java
new file mode 100644
index 000..9a832cc
--- /dev/null
+++ b/src/java/org/apache/cassandra/config/Converters.java
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.cassandra.config;
+
+import java.util.function.Function;
+
+/**
+ * Converters for backward compatibility with the old cassandra.yaml where
duration, data rate and
+ * data storage configuration parameters were provided only by value and the
expected unit was part of the configuration
+ * parameter name(suffix). (CASSANDRA-15234)
+ * It is important to be noted that this converter is not intended to be used
when we don't change name of a configuration
+ * parameter but we want to add unit. This would always default to the old
value provided without a unit at the moment.
+ * In case this functionality is needed at some point, please, raise a Jira
ticket.
+ */
+public enum Converters
+{
+/**
+ * This converter is used when we change the name of a cassandra.yaml
configuration parameter but we want to be
+ * able to still use the old name too. No units involved.
+ */
+IDENTITY(null, o -> o, o-> o),
+MILLIS_DURATION(Long.class,
+o -> DurationSpec.inMilliseconds((Long) o),
+o -> ((DurationSpec)o).toMilliseconds()),
+MILLIS_DOUBLE_DURATION(Double.class,
+ o -> DurationSpec.inDoubleMilliseconds((Double) o),
+ o -> ((DurationSpec)o).toMilliseconds()),
+/**
+ * This converter is used to support backward compatibility for parameters
where in the past -1 was used as a value
+ * Example: credentials_update_interval_in_ms = -1 and
credentials_update_interval = null (quantity of 0ms) are equal.
+ */
+MILLIS_CUSTOM_DURATION(Long.class,
+ o -> (Long)o == -1 ? new DurationSpec("0ms") :
DurationSpec.inMilliseconds((Long) o),
+ o -> ((DurationSpec)o).toMilliseconds() == 0 ? -1 :
((DurationSpec)o).toMilliseconds()),
+SECONDS_DURATION(Long.class,
+ o -> DurationSpec.inSeconds((Long) o),
+ o -> ((DurationSpec)o).toSeconds()),
+MINUTES_DURATION(Long.class,
+ o -> DurationSpec.inMinutes((Long) o),
+ o -> ((DurationSpec)o).toMinutes()),
+MEBIBYTES_DATA_STORAGE(Long.class,
+ o -> DataStorageSpec.inMebibytes((Long) o),
+ o -> ((DataStorageSpec)o).toMebibytes()),
+KIBIBYTES_DATASTORAGE(Long.class,
+ o -> DataStorageSpec.inKibibytes((Long) o),
+ o -> ((DataStorageSpec)o).toKibibytes()),
+BYTES_DATASTORAGE(Long.class,
+ o -> DataStorageSpec.inBytes((Long) o),
+ o -> ((DataStorageSpec)o).toBytes()),
+MEBIBYTES_PER_SECOND_DATA_RATE(Long.class,
+ o ->
DataRateSpec.inMebibytesPerSecond((Long) o),
+ o ->
((DataRateSpec)o).toMebibytesPerSecond()),
+/**
+ * This converter is a custom one to support backward compatibility for
stream_throughput_outbound and
+
[cassandra-dtest] branch trunk updated (0448f15 -> 3935906)
This is an automated email from the ASF dual-hosted git repository. edimitrova pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git. from 0448f15 When streaming sees a ClosedChannelException this triggers the disk failure policy add 3935906 Fixes needed to support the new configuration framework and change of parameters patch by Ekaterina Dimitrova, reviewed by Caleb Rackliffe, David Capwell, Michael Semb Wever and Benjamin Lerer for CASSANDRA-15234 No new revisions were added by this update. Summary of changes: offline_tools_test.py | 29 ++--- requirements.txt | 3 ++- tools/assertions.py | 5 - 3 files changed, 24 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17031) Add support for PEM based key material for SSL
[
https://issues.apache.org/jira/browse/CASSANDRA-17031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487537#comment-17487537
]
Stefan Miklosovic commented on CASSANDRA-17031:
---
Applied few fixes around examples (not compilable on Java 8 and fixed few
deprecated targets), going to merge this branch:
https://github.com/instaclustr/cassandra/tree/CASSANDRA-17031
> Add support for PEM based key material for SSL
> --
>
> Key: CASSANDRA-17031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17031
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Maulin Vasavada
>Assignee: Maulin Vasavada
>Priority: Normal
> Fix For: 4.1
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> h1. Scope
> Currently Cassandra supports standard keystore types for SSL
> keys/certificates. The scope of this enhancement is to add support for PEM
> based key material (keys/certificate) given that PEM is widely used common
> format for the same. We intend to add support for Unencrypted and Password
> Based Encrypted (PBE) PKCS#8 formatted Private Keys in PEM format with
> standard algorithms (RSA, DSA and EC) along with the certificate chain for
> the private key and PEM based X509 certificates. The work here is going to be
> built on top of [CEP-9: Make SSLContext creation
> pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable]
> for which the code is merged for Apache Cassandra 4.1 release.
> We intend to support the key material be configured as direct PEM values
> input OR via the file (configured with keystore and truststore configurations
> today). We are not going to model PEM as a valid 'store_type' given that
> 'store_type' has a [specific
> definition|https://docs.oracle.com/en/java/javase/11/security/java-cryptography-architecture-jca-reference-guide.html#GUID-AB51DEFD-5238-4F96-967F-082F6D34FBEA].
>
> h1. Approach
> Create an implementation for
> [ISslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/ISslContextFactory.java]
> extending
> [FileBasedSslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java]
> implementation to add PEM formatted key/certificates.
> h1. Motivation
> PEM is a widely used format for encoding Private Keys and X.509 Certificates
> and Apache Cassandra's current implementation lacks the support for
> specifying the PEM formatted key material for SSL configurations. This means
> operators have to re-create the key material to comply to the supported
> formats (using key/trust store types - jks, pkcs12 etc) and deal with an
> operational task for managing it. This is an operational overhead we can
> avoid by supporting the PEM format making Apache Cassandra even more customer
> friendly and drive more adoption.
> h1. Proposed Changes
> # A new implementation for ISslContextFactory - PEMBasedSslContextFactory
> with the following supported configuration
> {panel:title=New configurations}
> {panel}
> |{{encryption_options: }}
> {{}}{{ssl_context_factory:}}
> {{}}{{class_name:
> org.apache.cassandra.security.PEMBasedSslContextFactory}}
> {{}}{{parameters:}}
> {{ }}{{private_key: certificate chain>}}
> {{ }}{{private_key_password: }}{{private}} {{key }}{{if}} {{it is encrypted>}}
> {{ }}{{trusted_certificates: }}|
> *NOTE:* We could reuse 'keystore_password' instead of the
> 'private_key_password'. However PEM encoded private key is not a 'keystore'
> in itself hence it would be inappropriate to piggyback on that other than
> avoid duplicating similar fields.
> # The PEMBasedSslContextFactory will also support file based key material
> (and the corresponding HOT Reloading based on file timestamp updates) for the
> PEM format via existing 'keystore' and 'truststore' encryption options.
> However in that case the 'truststore_password' configuration won't be used
> since generally PEM formatted certificates for truststore don't get encrypted
> with a password.
> # The PEMBasedSslContextFactory will internally create PKCS12 keystore for
> private key and the trusted certificates. However, this doesn't impact the
> user of the implementation in anyway and it is mentioned for clarity only.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17031) Add support for PEM based key material for SSL
[
https://issues.apache.org/jira/browse/CASSANDRA-17031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-17031:
--
Reviewers: Jon Meredith, Stefan Miklosovic (was: Jon Meredith)
Status: Review In Progress (was: Patch Available)
> Add support for PEM based key material for SSL
> --
>
> Key: CASSANDRA-17031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17031
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Maulin Vasavada
>Assignee: Maulin Vasavada
>Priority: Normal
> Fix For: 4.1
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> h1. Scope
> Currently Cassandra supports standard keystore types for SSL
> keys/certificates. The scope of this enhancement is to add support for PEM
> based key material (keys/certificate) given that PEM is widely used common
> format for the same. We intend to add support for Unencrypted and Password
> Based Encrypted (PBE) PKCS#8 formatted Private Keys in PEM format with
> standard algorithms (RSA, DSA and EC) along with the certificate chain for
> the private key and PEM based X509 certificates. The work here is going to be
> built on top of [CEP-9: Make SSLContext creation
> pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable]
> for which the code is merged for Apache Cassandra 4.1 release.
> We intend to support the key material be configured as direct PEM values
> input OR via the file (configured with keystore and truststore configurations
> today). We are not going to model PEM as a valid 'store_type' given that
> 'store_type' has a [specific
> definition|https://docs.oracle.com/en/java/javase/11/security/java-cryptography-architecture-jca-reference-guide.html#GUID-AB51DEFD-5238-4F96-967F-082F6D34FBEA].
>
> h1. Approach
> Create an implementation for
> [ISslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/ISslContextFactory.java]
> extending
> [FileBasedSslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java]
> implementation to add PEM formatted key/certificates.
> h1. Motivation
> PEM is a widely used format for encoding Private Keys and X.509 Certificates
> and Apache Cassandra's current implementation lacks the support for
> specifying the PEM formatted key material for SSL configurations. This means
> operators have to re-create the key material to comply to the supported
> formats (using key/trust store types - jks, pkcs12 etc) and deal with an
> operational task for managing it. This is an operational overhead we can
> avoid by supporting the PEM format making Apache Cassandra even more customer
> friendly and drive more adoption.
> h1. Proposed Changes
> # A new implementation for ISslContextFactory - PEMBasedSslContextFactory
> with the following supported configuration
> {panel:title=New configurations}
> {panel}
> |{{encryption_options: }}
> {{}}{{ssl_context_factory:}}
> {{}}{{class_name:
> org.apache.cassandra.security.PEMBasedSslContextFactory}}
> {{}}{{parameters:}}
> {{ }}{{private_key: certificate chain>}}
> {{ }}{{private_key_password: }}{{private}} {{key }}{{if}} {{it is encrypted>}}
> {{ }}{{trusted_certificates: }}|
> *NOTE:* We could reuse 'keystore_password' instead of the
> 'private_key_password'. However PEM encoded private key is not a 'keystore'
> in itself hence it would be inappropriate to piggyback on that other than
> avoid duplicating similar fields.
> # The PEMBasedSslContextFactory will also support file based key material
> (and the corresponding HOT Reloading based on file timestamp updates) for the
> PEM format via existing 'keystore' and 'truststore' encryption options.
> However in that case the 'truststore_password' configuration won't be used
> since generally PEM formatted certificates for truststore don't get encrypted
> with a password.
> # The PEMBasedSslContextFactory will internally create PKCS12 keystore for
> private key and the trusted certificates. However, this doesn't impact the
> user of the implementation in anyway and it is mentioned for clarity only.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17031) Add support for PEM based key material for SSL
[
https://issues.apache.org/jira/browse/CASSANDRA-17031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-17031:
--
Status: Patch Available (was: In Progress)
https://github.com/apache/cassandra/pull/1316
> Add support for PEM based key material for SSL
> --
>
> Key: CASSANDRA-17031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17031
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Maulin Vasavada
>Assignee: Maulin Vasavada
>Priority: Normal
> Fix For: 4.1
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> h1. Scope
> Currently Cassandra supports standard keystore types for SSL
> keys/certificates. The scope of this enhancement is to add support for PEM
> based key material (keys/certificate) given that PEM is widely used common
> format for the same. We intend to add support for Unencrypted and Password
> Based Encrypted (PBE) PKCS#8 formatted Private Keys in PEM format with
> standard algorithms (RSA, DSA and EC) along with the certificate chain for
> the private key and PEM based X509 certificates. The work here is going to be
> built on top of [CEP-9: Make SSLContext creation
> pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable]
> for which the code is merged for Apache Cassandra 4.1 release.
> We intend to support the key material be configured as direct PEM values
> input OR via the file (configured with keystore and truststore configurations
> today). We are not going to model PEM as a valid 'store_type' given that
> 'store_type' has a [specific
> definition|https://docs.oracle.com/en/java/javase/11/security/java-cryptography-architecture-jca-reference-guide.html#GUID-AB51DEFD-5238-4F96-967F-082F6D34FBEA].
>
> h1. Approach
> Create an implementation for
> [ISslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/ISslContextFactory.java]
> extending
> [FileBasedSslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java]
> implementation to add PEM formatted key/certificates.
> h1. Motivation
> PEM is a widely used format for encoding Private Keys and X.509 Certificates
> and Apache Cassandra's current implementation lacks the support for
> specifying the PEM formatted key material for SSL configurations. This means
> operators have to re-create the key material to comply to the supported
> formats (using key/trust store types - jks, pkcs12 etc) and deal with an
> operational task for managing it. This is an operational overhead we can
> avoid by supporting the PEM format making Apache Cassandra even more customer
> friendly and drive more adoption.
> h1. Proposed Changes
> # A new implementation for ISslContextFactory - PEMBasedSslContextFactory
> with the following supported configuration
> {panel:title=New configurations}
> {panel}
> |{{encryption_options: }}
> {{}}{{ssl_context_factory:}}
> {{}}{{class_name:
> org.apache.cassandra.security.PEMBasedSslContextFactory}}
> {{}}{{parameters:}}
> {{ }}{{private_key: certificate chain>}}
> {{ }}{{private_key_password: }}{{private}} {{key }}{{if}} {{it is encrypted>}}
> {{ }}{{trusted_certificates: }}|
> *NOTE:* We could reuse 'keystore_password' instead of the
> 'private_key_password'. However PEM encoded private key is not a 'keystore'
> in itself hence it would be inappropriate to piggyback on that other than
> avoid duplicating similar fields.
> # The PEMBasedSslContextFactory will also support file based key material
> (and the corresponding HOT Reloading based on file timestamp updates) for the
> PEM format via existing 'keystore' and 'truststore' encryption options.
> However in that case the 'truststore_password' configuration won't be used
> since generally PEM formatted certificates for truststore don't get encrypted
> with a password.
> # The PEMBasedSslContextFactory will internally create PKCS12 keystore for
> private key and the trusted certificates. However, this doesn't impact the
> user of the implementation in anyway and it is mentioned for clarity only.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17031) Add support for PEM based key material for SSL
[
https://issues.apache.org/jira/browse/CASSANDRA-17031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-17031:
--
Status: Ready to Commit (was: Review In Progress)
> Add support for PEM based key material for SSL
> --
>
> Key: CASSANDRA-17031
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17031
> Project: Cassandra
> Issue Type: Improvement
> Components: Messaging/Internode
>Reporter: Maulin Vasavada
>Assignee: Maulin Vasavada
>Priority: Normal
> Fix For: 4.1
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> h1. Scope
> Currently Cassandra supports standard keystore types for SSL
> keys/certificates. The scope of this enhancement is to add support for PEM
> based key material (keys/certificate) given that PEM is widely used common
> format for the same. We intend to add support for Unencrypted and Password
> Based Encrypted (PBE) PKCS#8 formatted Private Keys in PEM format with
> standard algorithms (RSA, DSA and EC) along with the certificate chain for
> the private key and PEM based X509 certificates. The work here is going to be
> built on top of [CEP-9: Make SSLContext creation
> pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable]
> for which the code is merged for Apache Cassandra 4.1 release.
> We intend to support the key material be configured as direct PEM values
> input OR via the file (configured with keystore and truststore configurations
> today). We are not going to model PEM as a valid 'store_type' given that
> 'store_type' has a [specific
> definition|https://docs.oracle.com/en/java/javase/11/security/java-cryptography-architecture-jca-reference-guide.html#GUID-AB51DEFD-5238-4F96-967F-082F6D34FBEA].
>
> h1. Approach
> Create an implementation for
> [ISslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/ISslContextFactory.java]
> extending
> [FileBasedSslContextFactory|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/security/FileBasedSslContextFactory.java]
> implementation to add PEM formatted key/certificates.
> h1. Motivation
> PEM is a widely used format for encoding Private Keys and X.509 Certificates
> and Apache Cassandra's current implementation lacks the support for
> specifying the PEM formatted key material for SSL configurations. This means
> operators have to re-create the key material to comply to the supported
> formats (using key/trust store types - jks, pkcs12 etc) and deal with an
> operational task for managing it. This is an operational overhead we can
> avoid by supporting the PEM format making Apache Cassandra even more customer
> friendly and drive more adoption.
> h1. Proposed Changes
> # A new implementation for ISslContextFactory - PEMBasedSslContextFactory
> with the following supported configuration
> {panel:title=New configurations}
> {panel}
> |{{encryption_options: }}
> {{}}{{ssl_context_factory:}}
> {{}}{{class_name:
> org.apache.cassandra.security.PEMBasedSslContextFactory}}
> {{}}{{parameters:}}
> {{ }}{{private_key: certificate chain>}}
> {{ }}{{private_key_password: }}{{private}} {{key }}{{if}} {{it is encrypted>}}
> {{ }}{{trusted_certificates: }}|
> *NOTE:* We could reuse 'keystore_password' instead of the
> 'private_key_password'. However PEM encoded private key is not a 'keystore'
> in itself hence it would be inappropriate to piggyback on that other than
> avoid duplicating similar fields.
> # The PEMBasedSslContextFactory will also support file based key material
> (and the corresponding HOT Reloading based on file timestamp updates) for the
> PEM format via existing 'keystore' and 'truststore' encryption options.
> However in that case the 'truststore_password' configuration won't be used
> since generally PEM formatted certificates for truststore don't get encrypted
> with a password.
> # The PEMBasedSslContextFactory will internally create PKCS12 keystore for
> private key and the trusted certificates. However, this doesn't impact the
> user of the implementation in anyway and it is mentioned for clarity only.
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487532#comment-17487532
]
Stefan Miklosovic edited comment on CASSANDRA-16956 at 2/5/22, 5:53 PM:
https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05
https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch/1406/
was (Author: smiklosovic):
https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1
>
> Attachments: signature.asc, signature.asc, signature.asc,
> signature.asc, signature.asc, signature.asc, signature.asc, signature.asc
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16956:
--
Resolution: Fixed
Status: Resolved (was: Open)
https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.0.x, 4.x
>
> Attachments: signature.asc, signature.asc, signature.asc,
> signature.asc, signature.asc, signature.asc, signature.asc, signature.asc
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16956) Remove windows-specific classes
[
https://issues.apache.org/jira/browse/CASSANDRA-16956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-16956:
--
Fix Version/s: 4.1
(was: 4.x)
(was: 4.0.x)
> Remove windows-specific classes
> ---
>
> Key: CASSANDRA-16956
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16956
> Project: Cassandra
> Issue Type: Task
> Components: Build
>Reporter: Brandon Williams
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 4.1
>
> Attachments: signature.asc, signature.asc, signature.asc,
> signature.asc, signature.asc, signature.asc, signature.asc, signature.asc
>
>
> To continue the work CASSANDRA-16171 began, now that Windows support is no
> more there are some source files that can be removed.
> Just doing a naive grep on the source directory I see:
> {noformat}
> src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java
> src/java/org/apache/cassandra/utils/NativeLibraryWindows.java
> src/java/org/apache/cassandra/utils/WindowsTimer.java
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (28eea6e -> da47849)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 28eea6e Runtime-configurable YAML option to prohibit USE statements add da47849 Remove Windows-specific classes and related code No new revisions were added by this update. Summary of changes: .build/build-resolver.xml | 6 - CHANGES.txt| 1 + bin/cassandra | 8 -- bin/cqlsh.py | 48 ++-- bin/debug-cql | 8 -- conf/cassandra.yaml| 8 -- .../cassandra/pages/operating/security.adoc| 6 +- pylib/cqlshlib/copyutil.py | 12 +- pylib/cqlshlib/formatting.py | 2 - pylib/cqlshlib/test/run_cqlsh.py | 88 +++--- pylib/cqlshlib/test/test_cqlsh_completion.py | 6 +- pylib/cqlshlib/test/test_cqlsh_output.py | 13 +-- src/java/org/apache/cassandra/config/Config.java | 2 - .../cassandra/config/DatabaseDescriptor.java | 11 +- src/java/org/apache/cassandra/db/Directories.java | 6 +- .../cassandra/db/WindowsFailedSnapshotTracker.java | 117 --- .../db/commitlog/CommitLogSegmentManagerCDC.java | 1 - .../cassandra/db/compaction/CompactionManager.java | 6 +- .../cassandra/db/compaction/CompactionTask.java| 3 - .../apache/cassandra/db/lifecycle/LogReplica.java | 5 +- .../cassandra/db/lifecycle/LogTransaction.java | 7 +- .../db/streaming/CassandraOutgoingFile.java| 3 +- .../org/apache/cassandra/hints/HintsCatalog.java | 3 +- .../cassandra/io/sstable/SnapshotDeletingTask.java | 81 - .../io/sstable/metadata/MetadataSerializer.java| 5 - .../org/apache/cassandra/io/util/PathUtils.java| 4 - .../cassandra/net/InboundConnectionInitiator.java | 4 +- .../cassandra/repair/messages/RepairOption.java| 12 +- .../apache/cassandra/schema/SchemaConstants.java | 8 +- .../apache/cassandra/service/CassandraDaemon.java | 18 --- .../apache/cassandra/service/StartupChecks.java| 2 - .../apache/cassandra/service/StorageService.java | 7 -- .../org/apache/cassandra/tools/SSTableExport.java | 1 - .../org/apache/cassandra/utils/FBUtilities.java| 1 - .../org/apache/cassandra/utils/NativeLibrary.java | 7 +- .../cassandra/utils/NativeLibraryWindows.java | 127 - .../org/apache/cassandra/utils/SigarLibrary.java | 4 - .../org/apache/cassandra/utils/WindowsTimer.java | 69 --- test/conf/cdc.yaml | 3 - .../test/microbench/DirectorySizerBench.java | 5 - .../apache/cassandra/LogbackStatusListener.java| 8 -- .../unit/org/apache/cassandra/ServerTestUtils.java | 3 +- .../apache/cassandra/db/ColumnFamilyStoreTest.java | 5 - .../apache/cassandra/db/SystemKeyspaceTest.java| 36 +- .../db/commitlog/SnapshotDeletingTest.java | 107 - .../cassandra/db/lifecycle/LogTransactionTest.java | 3 +- .../cassandra/io/sstable/SSTableLoaderTest.java| 1 - .../cassandra/io/sstable/SSTableWriterTest.java| 33 ++ .../io/sstable/SSTableWriterTestBase.java | 10 -- .../repair/messages/RepairOptionTest.java | 8 +- .../service/StorageServiceServerTest.java | 74 .../apache/cassandra/utils/MonotonicClockTest.java | 4 +- .../src/org/apache/cassandra/stress/Stress.java| 13 +-- 53 files changed, 80 insertions(+), 953 deletions(-) delete mode 100644 src/java/org/apache/cassandra/db/WindowsFailedSnapshotTracker.java delete mode 100644 src/java/org/apache/cassandra/io/sstable/SnapshotDeletingTask.java delete mode 100644 src/java/org/apache/cassandra/utils/NativeLibraryWindows.java delete mode 100644 src/java/org/apache/cassandra/utils/WindowsTimer.java delete mode 100644 test/unit/org/apache/cassandra/db/commitlog/SnapshotDeletingTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15266) java internal exception on attempt to UPDATE a row using CONTAINS operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15266:
---
Description:
kostja@atlas ~ % cqlsh -ucassandra -pcassandra
Connected to My Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cassandra@cqlsh> CREATE KEYSPACE t1 WITH replication =
\{'class':'SimpleStrategy', 'replication_factor' : 1};
cassandra@cqlsh> use t1;
cassandra@cqlsh:t1> create table t (a int, b frozen>, c int,
primary key (a, b));
cassandra@cqlsh:t1> insert into t (a, b, c) values (1, \{1:1, 2:2}, 3);
cassandra@cqlsh:t1> update t set c=3 where a=1 and b contains 1;
ServerError: java.lang.UnsupportedOperationException
Server log file:
```
ERROR [Native-Transport-Requests-1] 2019-08-07 17:02:59,283
QueryMessage.java:129 - Unexpected error during query
java.lang.UnsupportedOperationException: null
at
org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$ContainsRestriction.appendTo(SingleColumnRestriction.java:454)
~[a
pache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.cql3.restrictions.ClusteringColumnRestrictions.valuesAsClustering(ClusteringColumnRestrictions.java:109)
~[a
pache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumns(StatementRestrictions.java:770)
~[apache-cassan
dra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.cql3.statements.ModificationStatement.createClustering(ModificationStatement.java:312)
~[apache-cassandra-3.
11.4.jar:3.11.4]
at
org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:677)
~[apache-cassandra-3.11.4.j
ar:3.11.4]
at
org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:635)
~[apache-cassandra-3.11.4
.jar:3.11.4]
at
org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:437)
~[apache-cassa
ndra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:425)
~[apache-cassandra-3.11.4.jar:
3.11.4]
at
org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:225)
~[apache-cassandra-3.11.4.jar:3.11.4]
at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
~[apache-cassandra-3.11.4.jar:3.11.4]
at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
~[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:566)
[apache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410)
[apache-cassandra-3.11.4.jar:3.11.4]
at
io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
[netty-all-4.0.44.Final.jar:4.0.44
...
```
+Additional information for newcomers:+
{{CONTAINS}} and {{CONTAINS KEY}} restrictions are not supported for {{UPDATE}}
or {{DELETE}} operations but they should be properly rejected with a proper
error message.
To fix that problem a new check should be added in the
{{StatementRestrictions}} constructor to thrown an {{InvalidRequestException}}
if the relation operator is a {{CONTAINS}} or {{CONTAINS_KEY}} and the
{{StatementType}} an {{UPDATE}} or a {{DELETION}}.
Some unit tests should be added to {{UpdateTest}} an {{DeleteTest}} to test the
behavior.
was:
kostja@atlas ~ % cqlsh -ucassandra -pcassandra
Connected to My Cluster at 127.0.0.1:9042.
[cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4]
Use HELP for help.
cassandra@cqlsh> CREATE KEYSPACE t1 WITH replication =
\{'class':'SimpleStrategy', 'replication_factor' : 1};
cassandra@cqlsh> use t1;
cassandra@cqlsh:t1> create table t (a int, b frozen>, c int,
primary key (a, b));
cassandra@cqlsh:t1> insert into t (a, b, c) values (1, \{1:1, 2:2}, 3);
cassandra@cqlsh:t1> update t set c=3 where a=1 and b contains 1;
ServerError: java.lang.UnsupportedOperationException
Server log file:
```
ERROR [Native-Transport-Requests-1] 2019-08-07 17:02:59,283
QueryMessage.java:129 - Unexpected error during query
java.lang.UnsupportedOperationException: null
at
org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$ContainsRestriction.appendTo(SingleColumnRestriction.java:454)
~[a
pache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.cql3.restrictions.ClusteringColumnRestrictions.valuesAsClustering(ClusteringColumnRestrictions.java:109)
~[a
pache-cassandra-3.11.4.jar:3.11.4]
at
org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumns(StatementRestrictions.java:770)
~[apache-cassan
dra-3.11.4.jar:3.11.4]
[jira] [Updated] (CASSANDRA-15266) java internal exception on attempt to UPDATE a row using CONTAINS operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15266:
---
Bug Category: Parent values: Code(13163)
Complexity: Low Hanging Fruit
Discovered By: User Report
Fix Version/s: 3.0.x
3.11.x
4.0.x
Mentor: Benjamin Lerer
Severity: Low
Status: Open (was: Triage Needed)
> java internal exception on attempt to UPDATE a row using CONTAINS operator
> --
>
> Key: CASSANDRA-15266
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15266
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Konstantin
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> kostja@atlas ~ % cqlsh -ucassandra -pcassandra
> Connected to My Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> cassandra@cqlsh> CREATE KEYSPACE t1 WITH replication =
> \{'class':'SimpleStrategy', 'replication_factor' : 1};
> cassandra@cqlsh> use t1;
> cassandra@cqlsh:t1> create table t (a int, b frozen>, c int,
> primary key (a, b));
> cassandra@cqlsh:t1> insert into t (a, b, c) values (1, \{1:1, 2:2}, 3);
> cassandra@cqlsh:t1> update t set c=3 where a=1 and b contains 1;
> ServerError: java.lang.UnsupportedOperationException
>
> Server log file:
> ```
> ERROR [Native-Transport-Requests-1] 2019-08-07 17:02:59,283
> QueryMessage.java:129 - Unexpected error during query
> java.lang.UnsupportedOperationException: null
> at
> org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$ContainsRestriction.appendTo(SingleColumnRestriction.java:454)
> ~[a
> pache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.restrictions.ClusteringColumnRestrictions.valuesAsClustering(ClusteringColumnRestrictions.java:109)
> ~[a
> pache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumns(StatementRestrictions.java:770)
> ~[apache-cassan
> dra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.createClustering(ModificationStatement.java:312)
> ~[apache-cassandra-3.
> 11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:677)
> ~[apache-cassandra-3.11.4.j
> ar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:635)
> ~[apache-cassandra-3.11.4
> .jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:437)
> ~[apache-cassa
> ndra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:425)
> ~[apache-cassandra-3.11.4.jar:
> 3.11.4]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:225)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:566)
> [apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410)
> [apache-cassandra-3.11.4.jar:3.11.4]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.44.Final.jar:4.0.44
> ...
> ```
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15266) java internal exception on attempt to UPDATE a row using CONTAINS operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-15266:
---
Labels: lhf (was: )
> java internal exception on attempt to UPDATE a row using CONTAINS operator
> --
>
> Key: CASSANDRA-15266
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15266
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/CQL
>Reporter: Konstantin
>Priority: Normal
> Labels: lhf
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> kostja@atlas ~ % cqlsh -ucassandra -pcassandra
> Connected to My Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.11.4 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> cassandra@cqlsh> CREATE KEYSPACE t1 WITH replication =
> \{'class':'SimpleStrategy', 'replication_factor' : 1};
> cassandra@cqlsh> use t1;
> cassandra@cqlsh:t1> create table t (a int, b frozen>, c int,
> primary key (a, b));
> cassandra@cqlsh:t1> insert into t (a, b, c) values (1, \{1:1, 2:2}, 3);
> cassandra@cqlsh:t1> update t set c=3 where a=1 and b contains 1;
> ServerError: java.lang.UnsupportedOperationException
>
> Server log file:
> ```
> ERROR [Native-Transport-Requests-1] 2019-08-07 17:02:59,283
> QueryMessage.java:129 - Unexpected error during query
> java.lang.UnsupportedOperationException: null
> at
> org.apache.cassandra.cql3.restrictions.SingleColumnRestriction$ContainsRestriction.appendTo(SingleColumnRestriction.java:454)
> ~[a
> pache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.restrictions.ClusteringColumnRestrictions.valuesAsClustering(ClusteringColumnRestrictions.java:109)
> ~[a
> pache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.restrictions.StatementRestrictions.getClusteringColumns(StatementRestrictions.java:770)
> ~[apache-cassan
> dra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.createClustering(ModificationStatement.java:312)
> ~[apache-cassandra-3.
> 11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.addUpdates(ModificationStatement.java:677)
> ~[apache-cassandra-3.11.4.j
> ar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.getMutations(ModificationStatement.java:635)
> ~[apache-cassandra-3.11.4
> .jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.executeWithoutCondition(ModificationStatement.java:437)
> ~[apache-cassa
> ndra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.cql3.statements.ModificationStatement.execute(ModificationStatement.java:425)
> ~[apache-cassandra-3.11.4.jar:
> 3.11.4]
> at
> org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:225)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:256)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:241)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.transport.messages.QueryMessage.execute(QueryMessage.java:116)
> ~[apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:566)
> [apache-cassandra-3.11.4.jar:3.11.4]
> at
> org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410)
> [apache-cassandra-3.11.4.jar:3.11.4]
> at
> io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
> [netty-all-4.0.44.Final.jar:4.0.44
> ...
> ```
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17350) Investigate and remove Windows-specific threading-related code in copyutils.py
[ https://issues.apache.org/jira/browse/CASSANDRA-17350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-17350: -- Description: There are bits of the code to be removed or refactored related to how Windows were treating threading in copyutils.py as Windows is not longer supported. [~Bowen Song] put it best so I just copy it here from GitHub PR for 16956, this code relates to FilesReader, FeedingProcess and ChildProcess Python classes. We agreed on the fact that 16956 may be merged without this being addressed as it requires further investigation in the matter which would unncessarily postpone and delay it. Bowen's take on this: I believe that we should move the code from on_fork() to __init__(), and may also remove the on_fork() method if it's no longer used. The problem with Windows and Python multiprocessing is that Windows doesn't support fork(), therefore Python implemented a workaround. On Windows, Python multiprocessing library uses pickle to serialise everything in memory, spawn a new process, and then restores the memory content from the serialised data. The ReceivingChannel and SendingChannel objects are not serialisable because they have file descriptors (which I believe it's called a file handle on Windows) in them, therefore the code has to handle them after the fake fork(). However, I'm concerned that moving the code from on_fork() to __init__() may have other unintended side effects. For example, the file descriptors (FDs) will be opened before the fork, in some edge cases the fork may never happen (e.g.: an exception raised in or just after the init, but before the fork). Where's the code responsible for closing the FDs on the parent process side? Will this cause any FD leak? This clearly requires further work to find out. To be honest, I don't think removing the comments without addressing the above is a wise move. Future developers wouldn't have the opportunity to understand why the code is written in this way if the comment is removed. was: There are bits of the code to be removed or refactored related to how Windows were treating threading in copyutils.py as Windows is not longer supported. FilesReader, FeedingProcess and ChildProcess: [~Bowen Song] put it the best so I just copy it here from GitHub PR for 16956, this code relates to FilesReader, FeedingProcess and ChildProcess Python classes. We agreed on the fact that 16956 may be merged without this being addressed as it requires further investigation in the matter which would unncessarily postpone and delay it. Bowen's take on this: I believe that we should move the code from on_fork() to __init__(), and may also remove the on_fork() method if it's no longer used. The problem with Windows and Python multiprocessing is that Windows doesn't support fork(), therefore Python implemented a workaround. On Windows, Python multiprocessing library uses pickle to serialise everything in memory, spawn a new process, and then restores the memory content from the serialised data. The ReceivingChannel and SendingChannel objects are not serialisable because they have file descriptors (which I believe it's called a file handle on Windows) in them, therefore the code has to handle them after the fake fork(). However, I'm concerned that moving the code from on_fork() to __init__() may have other unintended side effects. For example, the file descriptors (FDs) will be opened before the fork, in some edge cases the fork may never happen (e.g.: an exception raised in or just after the init, but before the fork). Where's the code responsible for closing the FDs on the parent process side? Will this cause any FD leak? This clearly requires further work to find out. To be honest, I don't think removing the comments without addressing the above is a wise move. Future developers wouldn't have the opportunity to understand why the code is written in this way if the comment is removed. > Investigate and remove Windows-specific threading-related code in copyutils.py > -- > > Key: CASSANDRA-17350 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17350 > Project: Cassandra > Issue Type: Improvement > Components: Tool/cqlsh >Reporter: Stefan Miklosovic >Priority: Normal > > There are bits of the code to be removed or refactored related to how Windows > were treating threading in copyutils.py as Windows is not longer supported. > [~Bowen Song] put it best so I just copy it here from GitHub PR for 16956, > this code relates to FilesReader, FeedingProcess and ChildProcess Python > classes. > We agreed on the fact that 16956 may be merged without this being addressed > as it requires further investigation in the matter which would
[jira] [Created] (CASSANDRA-17350) Investigate and remove Windows-specific threading-related code in copyutils.py
Stefan Miklosovic created CASSANDRA-17350: - Summary: Investigate and remove Windows-specific threading-related code in copyutils.py Key: CASSANDRA-17350 URL: https://issues.apache.org/jira/browse/CASSANDRA-17350 Project: Cassandra Issue Type: Improvement Components: Tool/cqlsh Reporter: Stefan Miklosovic There are bits of the code to be removed or refactored related to how Windows were treating threading in copyutils.py as Windows is not longer supported. FilesReader, FeedingProcess and ChildProcess: [~Bowen Song] put it the best so I just copy it here from GitHub PR for 16956, this code relates to FilesReader, FeedingProcess and ChildProcess Python classes. We agreed on the fact that 16956 may be merged without this being addressed as it requires further investigation in the matter which would unncessarily postpone and delay it. Bowen's take on this: I believe that we should move the code from on_fork() to __init__(), and may also remove the on_fork() method if it's no longer used. The problem with Windows and Python multiprocessing is that Windows doesn't support fork(), therefore Python implemented a workaround. On Windows, Python multiprocessing library uses pickle to serialise everything in memory, spawn a new process, and then restores the memory content from the serialised data. The ReceivingChannel and SendingChannel objects are not serialisable because they have file descriptors (which I believe it's called a file handle on Windows) in them, therefore the code has to handle them after the fake fork(). However, I'm concerned that moving the code from on_fork() to __init__() may have other unintended side effects. For example, the file descriptors (FDs) will be opened before the fork, in some edge cases the fork may never happen (e.g.: an exception raised in or just after the init, but before the fork). Where's the code responsible for closing the FDs on the parent process side? Will this cause any FD leak? This clearly requires further work to find out. To be honest, I don't think removing the comments without addressing the above is a wise move. Future developers wouldn't have the opportunity to understand why the code is written in this way if the comment is removed. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[
https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487494#comment-17487494
]
Benedict Elliott Smith edited comment on CASSANDRA-17292 at 2/5/22, 1:57 PM:
-
bq. if you are looking at that and new to Cassandra, will you think validation
is related to compaction? What about repair?
None of these are in my proposed layout file, in fact there is no separate
validation compaction throughput limiter that I can see in either proposal, or
your dump? In my proposal I see
{code}
throughput:
streaming:
local: 25MiB/s
remote: 25MiB/s
batchlog: 1MiB/s# total for node; peers receive proportional
share
compaction: 16MiB/s
hint_delivery: 1MiB/s
{code}
If you wanted to list a separate validation compaction limiter, I would
probably call it e.g. {{compaction_for_repair}}. Today the
{{concurrent_validations}} is a much better example of something that makes no
sense already to a user without pre-existing knowledge, despite its partial
context.
was (Author: benedict):
bq. if you are looking at that and new to Cassandra, will you think validation
is related to compaction? What about repair?
None of these are in my proposed layout file, in fact there is no separate
validation compaction throughput limiter that I can see? In my proposal I see
{code}
throughput:
streaming:
local: 25MiB/s
remote: 25MiB/s
batchlog: 1MiB/s# total for node; peers receive proportional
share
compaction: 16MiB/s
hint_delivery: 1MiB/s
{code}
If you wanted to list a separate validation compaction limiter, I would
probably call it e.g. {{compaction_for_repair}}. Today the
{{concurrent_validations}} is a much better example of something that makes no
sense already to a user without pre-existing knowledge, despite its partial
context.
> Move cassandra.yaml toward a nested structure around major database concepts
>
>
> Key: CASSANDRA-17292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17292
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.x
>
>
> Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new
> features") has made it clear we will gravitate toward appropriately nested
> structures for new parameters in {{cassandra.yaml}}, but from the scattered
> conversation across a few Guardrails tickets (see CASSANDRA-17212 and
> CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to
> eventually extend this to the rest of {{cassandra.yaml}}. The benefits of
> this change include those we gain by doing it for new features (single point
> of interest for feature documentation, typed configuration objects, logical
> grouping for additional parameters added over time, discoverability, etc.),
> but one a larger scale.
> This may overlap with ongoing work, including the Guardrails epic. Ideally,
> even a rough cut of a design here would allow that to move forward in a
> timely and coherent manner (with less long-term refactoring pain).
> While these would have to be adjusted to CASSANDRA-15234 (probably after it
> merges), there have been two proposals floated already for what this might
> look like:
> From [~maedhroz] -
> https://github.com/maedhroz/cassandra/commit/49e83c70eba3357978d1081ecf500bbbdee960d8
> From [~benedict] -
> https://github.com/belliottsmith/cassandra/commits/CASSANDRA-15234-grouping-ideas
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[
https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487494#comment-17487494
]
Benedict Elliott Smith commented on CASSANDRA-17292:
bq. if you are looking at that and new to Cassandra, will you think validation
is related to compaction? What about repair?
None of these are in my proposed layout file, in fact there is no separate
validation compaction throughput limiter that I can see? In my proposal I see
{code}
throughput:
streaming:
local: 25MiB/s
remote: 25MiB/s
batchlog: 1MiB/s# total for node; peers receive proportional
share
compaction: 16MiB/s
hint_delivery: 1MiB/s
{code}
If you wanted to list a separate validation compaction limiter, I would
probably call it e.g. {{compaction_for_repair}}. Today the
{{concurrent_validations}} is a much better example of something that makes no
sense already to a user without pre-existing knowledge, despite its partial
context.
> Move cassandra.yaml toward a nested structure around major database concepts
>
>
> Key: CASSANDRA-17292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17292
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.x
>
>
> Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new
> features") has made it clear we will gravitate toward appropriately nested
> structures for new parameters in {{cassandra.yaml}}, but from the scattered
> conversation across a few Guardrails tickets (see CASSANDRA-17212 and
> CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to
> eventually extend this to the rest of {{cassandra.yaml}}. The benefits of
> this change include those we gain by doing it for new features (single point
> of interest for feature documentation, typed configuration objects, logical
> grouping for additional parameters added over time, discoverability, etc.),
> but one a larger scale.
> This may overlap with ongoing work, including the Guardrails epic. Ideally,
> even a rough cut of a design here would allow that to move forward in a
> timely and coherent manner (with less long-term refactoring pain).
> While these would have to be adjusted to CASSANDRA-15234 (probably after it
> merges), there have been two proposals floated already for what this might
> look like:
> From [~maedhroz] -
> https://github.com/maedhroz/cassandra/commit/49e83c70eba3357978d1081ecf500bbbdee960d8
> From [~benedict] -
> https://github.com/belliottsmith/cassandra/commits/CASSANDRA-15234-grouping-ideas
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[
https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487476#comment-17487476
]
Benedict Elliott Smith edited comment on CASSANDRA-17292 at 2/5/22, 1:33 PM:
-
bq. At the moment streaming and compaction are configured separately
We have a largely flat and messy config file today, so I don't think what we do
today is relevant. Streaming and compaction are intrinsically linked by repair
(except in the case of bootstrap). Streaming is gated by (validation)
compaction throughput. Where does repair configuration sit in this world? Where
should streaming network settings sit?
You also really need to address the logical inconsistency of
{{materialized_views.concurrent_writes}} and {{query.concurrent_writes}}, or
{{materialized_views.enabled}} and {{query.enable_user_defined_functions}}. In
each case we have semantically equivalent things dotted in inconsistent config
(if {{concurrent_writes}} is a query option, so is
{{concurrent_materialized_view_writes}}; if {{enable_user_defined_functions}}
is a query/cql option so is {{enable_materialized_views}}).
Honestly, if we cannot come up with a _coherent_ strategy that avoids the above
inconsistencies I prefer the grab bag of flat config we have today, just tidied
up a bit. Nesting inconsistently is strictly worse for usability IMO.
bq. I have never worked on a project where I didn't ask how to configure a
feature or a subsystem and instead wanted to look at all rate limiters together
You have never had to address database behaviour concerns that cut across
features?
In my stint as a database operator, most configuration was of no interest. I
did not typically delve into feature-level configuration. What I was interested
in is what settings I needed to set for it to operate correctly, what settings
might affect the database performance, and what settings might affect security
or other stability concerns. I would absolutely have preferred to see them
presented together rather than spread across the many features I did not know
of or understand.
bq. but if we do actually implement pluggable storage, where will this be?
This same argument can likely be applied to concurrent_reads and
concurrent_writes - it also applies to commit log (and implicitly CDC), repair,
streaming, hints, memtables and compaction. Even many of the guardrails,
particularly e.g. involving tombstones (which are a storage layer concept not
all implementations will share). Even MVs perhaps (due to special tombstones).
Are we proposing to group these all under {{storage}}?
IMO {{storage}} and {{query}} are such broad terms that almost everything can
be justified as encompassed by them. To me this is poor API design, as the user
has to guess what the authors were thinking, whether in this case it went under
this heading, or that one, or if this one was important enough it got its own
heading. Particularly if the user doesn't know a priori what the possible
configuration options are.
was (Author: benedict):
bq. At the moment streaming and compaction are configured separately
We have a largely flat and messy config file today, so I don't think what we do
today is relevant. Streaming and compaction are intrinsically linked by repair
(except in the case of bootstrap). Streaming is gated by (validation)
compaction throughput. Where does repair configuration sit in this world? Where
should streaming network settings sit?
You also really need to address the logical inconsistency of
{{materialized_views.concurrent_writes}} and {{query.concurrent_writes}}, or
{{materialized_views.enabled}} and {{query.enable_user_defined_functions}}. In
each case we have semantically equivalent things dotted in inconsistent config
(if {{concurrent_writes}} is a query option, so is
{{concurrent_materialized_view_writes}}; if {{enable_user_defined_functions}}
is a query/cql option so is {{enable_materialized_views}}).
Honestly, if we cannot come up with a _coherent_ strategy that avoids the above
inconsistencies I prefer the grab bag of flat config we have today, just tidied
up a bit. Nesting inconsistently is strictly worse for usability IMO.
bq. I have never worked on a project where I didn't ask how to configure a
feature or a subsystem and instead wanted to look at all rate limiters together
You have never had to address database behaviour concerns that cut across
features? In my stint as a database operator, most configuration was of no
interest. I did not typically delve into feature-level configuration. System
settings, tuning and security are the only things I would be interested in, and
I would absolutely have preferred to see them presented together rather than
spread across the many features I did not know of or understand.
bq. but if we do actually implement pluggable storage, where will
[jira] [Comment Edited] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[
https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487476#comment-17487476
]
Benedict Elliott Smith edited comment on CASSANDRA-17292 at 2/5/22, 1:31 PM:
-
bq. At the moment streaming and compaction are configured separately
We have a largely flat and messy config file today, so I don't think what we do
today is relevant. Streaming and compaction are intrinsically linked by repair
(except in the case of bootstrap). Streaming is gated by (validation)
compaction throughput. Where does repair configuration sit in this world? Where
should streaming network settings sit?
You also really need to address the logical inconsistency of
{{materialized_views.concurrent_writes}} and {{query.concurrent_writes}}, or
{{materialized_views.enabled}} and {{query.enable_user_defined_functions}}. In
each case we have semantically equivalent things dotted in inconsistent config
(if {{concurrent_writes}} is a query option, so is
{{concurrent_materialized_view_writes}}; if {{enable_user_defined_functions}}
is a query/cql option so is {{enable_materialized_views}}).
Honestly, if we cannot come up with a _coherent_ strategy that avoids the above
inconsistencies I prefer the grab bag of flat config we have today, just tidied
up a bit. Nesting inconsistently is strictly worse for usability IMO.
bq. I have never worked on a project where I didn't ask how to configure a
feature or a subsystem and instead wanted to look at all rate limiters together
You have never had to address database behaviour concerns that cut across
features? In my stint as a database operator, most configuration was of no
interest. I did not typically delve into feature-level configuration. System
settings, tuning and security are the only things I would be interested in, and
I would absolutely have preferred to see them presented together rather than
spread across the many features I did not know of or understand.
bq. but if we do actually implement pluggable storage, where will this be?
This same argument can likely be applied to concurrent_reads and
concurrent_writes - it also applies to commit log (and implicitly CDC), repair,
streaming, hints, memtables and compaction. Even many of the guardrails,
particularly e.g. involving tombstones (which are a storage layer concept not
all implementations will share). Even MVs perhaps (due to special tombstones).
Are we proposing to group these all under {{storage}}?
IMO {{storage}} and {{query}} are such broad terms that almost everything can
be justified as encompassed by them. To me this is poor API design, as the user
has to guess what the authors were thinking, whether in this case it went under
this heading, or that one, or if this one was important enough it got its own
heading. Particularly if the user doesn't know a priori what the possible
configuration options are.
was (Author: benedict):
bq. At the moment streaming and compaction are configured separately
We have a largely flat and messy config file today, so I don't think what we do
today is relevant. Streaming and compaction are intrinsically linked by repair
(except in the case of bootstrap). Streaming is gated by compaction throughput.
Where does repair configuration sit in this world? Where should streaming
network configurations sit?
You also haven't addressed the clear inconsistency of
{{materialized_views.concurrent_writes}} and {{query.concurrent_writes}}, or
{{materialized_views.enabled}} and {{query.enable_user_defined_functions}}. In
each case we have semantically equivalent things dotted in inconsistent config
(if {{concurrent_writes}} is a query option, so is
{{concurrent_materialized_view_writes}}; if {{enable_user_defined_functions}}
is a query/cql option so is {{enable_materialized_views}}).
Honestly, if we cannot come up with a _coherent_ strategy that avoids the above
inconsistencies I prefer the grab bag of flat config we have today, just tidied
up a bit. Nesting inconsistently is strictly worse for usability IMO.
bq. I have never worked on a project where I didn't ask how to configure a
feature or a subsystem and instead wanted to look at all rate limiters together
You have never had to address database behaviour concerns that cut across
features?
bq. but if we do actually implement pluggable storage, where will this be?
This same argument can likely be applied to concurrent_reads and
concurrent_writes - it also applies to commit log (and implicitly CDC), repair,
streaming, hints, memtables and compaction. Are we going to group these all
under storage?
> Move cassandra.yaml toward a nested structure around major database concepts
>
>
> Key: CASSANDRA-17292
> URL:
[jira] [Updated] (CASSANDRA-17236) Add support for short form of version to CQLSH
[
https://issues.apache.org/jira/browse/CASSANDRA-17236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-17236:
-
Reviewers: Brandon Williams
> Add support for short form of version to CQLSH
> --
>
> Key: CASSANDRA-17236
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17236
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Semantics
>Reporter: Madhavan
>Assignee: Yash Ladha
>Priority: Low
> Labels: lhf
> Fix For: 4.x
>
>
> Today we do only support the `–version` long option/form for cqlsh and this
> enhancement Jira is to request that we also offer a shorter version `-v` to
> cqlsh. This will have consistency benefits with other tools and even match
> with what we have at `bin/cassandra -v` option for instance.
> Today, `cqlsh` does support `--v` to get the version which is different than
> the single dashed short form that is available at many other tools. Thanks to
> Ekaterina for finding this. It looks like this is stemming from Python's
> parse mechanism which is detailed here,
> [https://docs.python.org/2.7/library/optparse.html#printing-a-version-string].
>
> [https://github.com/apache/cassandra/blob/trunk/bin/cqlsh.py#L194-L196]
> {quote}parser = optparse.OptionParser(description=description, epilog=epilog,
> usage="Usage: %prog [options] [host [port]]",
> version='cqlsh ' + version)
> {quote}
>
> {{$ bin/cqlsh --v}}
> {{cqlsh 6.0.0}}
> This looks like a weird implementation at Python. Both (\-\-help) and
> (\-\-version) options are stemming from here,
> [https://github.com/python/cpython/blob/2.7/Lib/optparse.py#L1248-L1256] and
> they did decide to ignore the short form option for version and it somehow
> automatically takes the (--v) option to spit the version info.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated: Add versions logging for build tools
This is an automated email from the ASF dual-hosted git repository.
azotcsit pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/trunk by this push:
new 51a654f Add versions logging for build tools
51a654f is described below
commit 51a654f0bddc6d8a0db03132698dd5229f425752
Author: Aleksei Zotov
AuthorDate: Wed Feb 2 21:08:24 2022 +0400
Add versions logging for build tools
patch by Aleksei Zotov; reviewed by Mick Semb Wever for CASSANDRA-16630
---
build-scripts/cassandra-artifacts.sh | 5 +
build-scripts/cassandra-dtest-pytest-docker.sh | 6 ++
build-scripts/cassandra-dtest-pytest.sh| 10 ++
build-scripts/cassandra-test-docker.sh | 3 +++
build-scripts/cassandra-test.sh| 4
5 files changed, 28 insertions(+)
diff --git a/build-scripts/cassandra-artifacts.sh
b/build-scripts/cassandra-artifacts.sh
index 80185f9..07dd620 100755
--- a/build-scripts/cassandra-artifacts.sh
+++ b/build-scripts/cassandra-artifacts.sh
@@ -18,6 +18,11 @@ command -v docker >/dev/null 2>&1 || { echo >&2 "docker
needs to be installed";
[ -f "build.xml" ] || { echo >&2 "build.xml must exist"; exit 1; }
[ -d "${cassandra_builds_dir}" ] || { echo >&2 "cassandra-builds directory
must exist"; exit 1; }
+# print debug information on versions
+ant -version
+pip --version
+virtualenv --version
+docker --version
# Sphinx is needed for the gen-doc target
virtualenv venv
diff --git a/build-scripts/cassandra-dtest-pytest-docker.sh
b/build-scripts/cassandra-dtest-pytest-docker.sh
index d931961..5ff3cc1 100755
--- a/build-scripts/cassandra-dtest-pytest-docker.sh
+++ b/build-scripts/cassandra-dtest-pytest-docker.sh
@@ -43,6 +43,12 @@ DTEST_REPO=$3
DTEST_BRANCH=$4
EOF
+# pre-conditions
+command -v docker >/dev/null 2>&1 || { echo >&2 "docker needs to be
installed"; exit 1; }
+
+# print debug information on versions
+docker --version
+
set -x # debug, sometimes ${docker_cpus} is not evaluated
# Jenkins agents run multiple executors per machine. `jenkins_executors=1`
is used for anything non-jenkins.
jenkins_executors=1
diff --git a/build-scripts/cassandra-dtest-pytest.sh
b/build-scripts/cassandra-dtest-pytest.sh
index 89eb59b..448b50c 100755
--- a/build-scripts/cassandra-dtest-pytest.sh
+++ b/build-scripts/cassandra-dtest-pytest.sh
@@ -31,6 +31,16 @@ if [ $? -eq 0 -a -n "$JAVA8_HOME" -a -n "$JAVA11_HOME" ];
then
export JAVA_HOME="$JAVA11_HOME"
fi
+# pre-conditions
+command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to be installed"; exit
1; }
+command -v pip3 >/dev/null 2>&1 || { echo >&2 "pip3 needs to be installed";
exit 1; }
+command -v virtualenv >/dev/null 2>&1 || { echo >&2 "virtualenv needs to be
installed"; exit 1; }
+
+# print debug information on versions
+ant -version
+pip3 --version
+virtualenv --version
+
# Loop to prevent failure due to maven-ant-tasks not downloading a jar..
for x in $(seq 1 3); do
ant clean jar
diff --git a/build-scripts/cassandra-test-docker.sh
b/build-scripts/cassandra-test-docker.sh
index 254c1a9..2693881 100755
--- a/build-scripts/cassandra-test-docker.sh
+++ b/build-scripts/cassandra-test-docker.sh
@@ -35,6 +35,9 @@ else
command -v docker >/dev/null 2>&1 || { echo >&2 "docker needs to be
installed"; exit 1; }
(docker info >/dev/null 2>&1) || { echo >&2 "docker needs to running";
exit 1; }
+# print debug information on versions
+docker --version
+
# start the docker container
if [ "$#" -lt 5 ]; then
echo "Usage: cassandra-test-docker.sh REPO BRANCH BUILDS_REPO_URL
BUILDS_BRANCH DOCKER_IMAGE [target] [split_chunk]"
diff --git a/build-scripts/cassandra-test.sh b/build-scripts/cassandra-test.sh
index d783d86..2605b08 100755
--- a/build-scripts/cassandra-test.sh
+++ b/build-scripts/cassandra-test.sh
@@ -10,6 +10,10 @@ command -v ant >/dev/null 2>&1 || { echo >&2 "ant needs to
be installed"; exit 1
command -v git >/dev/null 2>&1 || { echo >&2 "git needs to be installed"; exit
1; }
[ -f "build.xml" ] || { echo >&2 "build.xml must exist"; exit 1; }
+# print debug information on versions
+ant -version
+git --version
+
# lists all tests for the specific test type
_list_tests() {
local -r classlistprefix="$1"
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[
https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487476#comment-17487476
]
Benedict Elliott Smith edited comment on CASSANDRA-17292 at 2/5/22, 11:49 AM:
--
bq. At the moment streaming and compaction are configured separately
We have a largely flat and messy config file today, so I don't think what we do
today is relevant. Streaming and compaction are intrinsically linked by repair
(except in the case of bootstrap). Streaming is gated by compaction throughput.
Where does repair configuration sit in this world? Where should streaming
network configurations sit?
You also haven't addressed the clear inconsistency of
{{materialized_views.concurrent_writes}} and {{query.concurrent_writes}}, or
{{materialized_views.enabled}} and {{query.enable_user_defined_functions}}. In
each case we have semantically equivalent things dotted in inconsistent config
(if {{concurrent_writes}} is a query option, so is
{{concurrent_materialized_view_writes}}; if {{enable_user_defined_functions}}
is a query/cql option so is {{enable_materialized_views}}).
Honestly, if we cannot come up with a _coherent_ strategy that avoids the above
inconsistencies I prefer the grab bag of flat config we have today, just tidied
up a bit. Nesting inconsistently is strictly worse for usability IMO.
bq. I have never worked on a project where I didn't ask how to configure a
feature or a subsystem and instead wanted to look at all rate limiters together
You have never had to address database behaviour concerns that cut across
features?
bq. but if we do actually implement pluggable storage, where will this be?
This same argument can likely be applied to concurrent_reads and
concurrent_writes - it also applies to commit log (and implicitly CDC), repair,
streaming, hints, memtables and compaction. Are we going to group these all
under storage?
was (Author: benedict):
bq. At the moment streaming and compaction are configured separately
We have a largely flat and messy config file today, so I don't think what we do
today is relevant. Streaming and compaction are intrinsically linked by repair
(except in the case of bootstrap). Streaming is gated by compaction throughput.
Where does repair configuration sit in this world? Where should streaming
network configurations sit?
You also haven't addressed the clear inconsistency of
{{materialized_views.concurrent_writes}} and {{query.concurrent_writes}}, or
{{materialized_views.enabled}} and {{query.enable_user_defined_functions}}. In
each case we have semantically equivalent things dotted in entirely unrelated
config.
Honestly, if we cannot come up with a _coherent_ strategy that avoids the above
inconsistencies I prefer the grab bag of flat config we have today, just tidied
up a bit. Nesting inconsistently is strictly worse for usability IMO.
bq. I have never worked on a project where I didn't ask how to configure a
feature or a subsystem and instead wanted to look at all rate limiters together
You have never had to address database behaviour concerns that cut across
features?
bq. but if we do actually implement pluggable storage, where will this be?
This same argument can likely be applied to concurrent_reads and
concurrent_writes - it also applies to commit log (and implicitly CDC), repair,
streaming, hints, memtables and compaction. Are we going to group these all
under storage?
> Move cassandra.yaml toward a nested structure around major database concepts
>
>
> Key: CASSANDRA-17292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17292
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.x
>
>
> Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new
> features") has made it clear we will gravitate toward appropriately nested
> structures for new parameters in {{cassandra.yaml}}, but from the scattered
> conversation across a few Guardrails tickets (see CASSANDRA-17212 and
> CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to
> eventually extend this to the rest of {{cassandra.yaml}}. The benefits of
> this change include those we gain by doing it for new features (single point
> of interest for feature documentation, typed configuration objects, logical
> grouping for additional parameters added over time, discoverability, etc.),
> but one a larger scale.
> This may overlap with ongoing work, including the Guardrails epic. Ideally,
> even a rough cut of a design here would allow that to move forward in a
> timely and coherent manner (with
[jira] [Commented] (CASSANDRA-17292) Move cassandra.yaml toward a nested structure around major database concepts
[
https://issues.apache.org/jira/browse/CASSANDRA-17292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17487476#comment-17487476
]
Benedict Elliott Smith commented on CASSANDRA-17292:
bq. At the moment streaming and compaction are configured separately
We have a largely flat and messy config file today, so I don't think what we do
today is relevant. Streaming and compaction are intrinsically linked by repair
(except in the case of bootstrap). Streaming is gated by compaction throughput.
Where does repair configuration sit in this world? Where should streaming
network configurations sit?
You also haven't addressed the clear inconsistency of
{{materialized_views.concurrent_writes}} and {{query.concurrent_writes}}, or
{{materialized_views.enabled}} and {{query.enable_user_defined_functions}}. In
each case we have semantically equivalent things dotted in entirely unrelated
config.
Honestly, if we cannot come up with a _coherent_ strategy that avoids the above
inconsistencies I prefer the grab bag of flat config we have today, just tidied
up a bit. Nesting inconsistently is strictly worse for usability IMO.
bq. I have never worked on a project where I didn't ask how to configure a
feature or a subsystem and instead wanted to look at all rate limiters together
You have never had to address database behaviour concerns that cut across
features?
bq. but if we do actually implement pluggable storage, where will this be?
This same argument can likely be applied to concurrent_reads and
concurrent_writes - it also applies to commit log (and implicitly CDC), repair,
streaming, hints, memtables and compaction. Are we going to group these all
under storage?
> Move cassandra.yaml toward a nested structure around major database concepts
>
>
> Key: CASSANDRA-17292
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17292
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Caleb Rackliffe
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.x
>
>
> Recent mailing list conversation (see "[DISCUSS] Nested YAML configs for new
> features") has made it clear we will gravitate toward appropriately nested
> structures for new parameters in {{cassandra.yaml}}, but from the scattered
> conversation across a few Guardrails tickets (see CASSANDRA-17212 and
> CASSANDRA-17148) and CASSANDRA-15234, there is also a general desire to
> eventually extend this to the rest of {{cassandra.yaml}}. The benefits of
> this change include those we gain by doing it for new features (single point
> of interest for feature documentation, typed configuration objects, logical
> grouping for additional parameters added over time, discoverability, etc.),
> but one a larger scale.
> This may overlap with ongoing work, including the Guardrails epic. Ideally,
> even a rough cut of a design here would allow that to move forward in a
> timely and coherent manner (with less long-term refactoring pain).
> While these would have to be adjusted to CASSANDRA-15234 (probably after it
> merges), there have been two proposals floated already for what this might
> look like:
> From [~maedhroz] -
> https://github.com/maedhroz/cassandra/commit/49e83c70eba3357978d1081ecf500bbbdee960d8
> From [~benedict] -
> https://github.com/belliottsmith/cassandra/commits/CASSANDRA-15234-grouping-ideas
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
