[
https://issues.apache.org/jira/browse/CASSANDRA-20249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18072972#comment-18072972
]
Pedro Gordo edited comment on CASSANDRA-20249 at 4/12/26 3:24 PM:
------------------------------------------------------------------
I've been working on this ticket — I have a JUnit test
(\{{ConfigYamlCoverageTest}}) with forward and reverse checks, and an
\{{@HiddenInYaml}} annotation ready to go. I've triaged all 131 Config.java
fields that are missing from the yaml files and have decisions suggestions for
most of them.
Before I share the full spreadsheet, I'd like to get a quick sense of whether
we can handle some groups at the group level rather than property-by-property:
* *Deprecated fields (13)*: \{{back_pressure_*}}, \{{otc_coalescing_*}},
\{{otc_backlog_*}}, \{{windows_timer_interval}}, \{{max_streaming_retries}},
\{{repair_session_max_tree_depth}},
\{{scripted_user_defined_functions_enabled}}, \{{use_deterministic_table_id}},
\{{cms_default_retry_backoff}}, \{{cms_default_max_retry_backoff}}. These are
all \{{@Deprecated}} in Config.java. The test already skips them automatically,
but should they also be annotated \{{@HiddenInYaml}} for clarity?
* *Paxos v2 (16 fields)*: All the \{{paxos_*}} and \{{skip_paxos_repair_*}}
fields from CEP-14. Internal LWT protocol tuning.
* *TCM/CMS (12 fields)*: CMS timeouts/retries (\{{cms_*}}), metadata
snapshotting, discovery timeout, \{{unsafe_tcm_mode}}, progress barrier fields
(\{{progress_barrier_*}}), and \{{short_rpc_timeout}}. All internal CEP-21
infrastructure.
* *Accord (3 fields)*: \{{accord_preaccept_timeout}},
\{{concurrent_accord_operations}}, \{{consensus_migration_cache_size}}. CEP-15
internals, still maturing.
Does anyone see a reason any of these should be exposed in cassandra.yaml
rather than marked \{{@HiddenInYaml}}? If we can agree on these groups, it
reduces the remaining discussion to about 13 individual fields where there are
differing opinions.
was (Author: pedro_gordo):
I've been working on this ticket — I have a JUnit test
({{ConfigYamlCoverageTest}}) with forward and reverse checks, and an
{{@HiddenInYaml}} annotation ready to go. I've triaged all 131 Config.java
fields that are missing from the yaml files and have decisions suggestions for
most of them.
Before I share the full spreadsheet, I'd like to get a quick sense of whether
we can handle some groups at the group level rather than property-by-property:
- \*Deprecated fields (13)\*: {{back\_pressure\_\*}}, {{otc\_coalescing\_\*}},
{{otc\_backlog\_\*}}, {{windows\_timer\_interval}},
{{max\_streaming\_retries}}, {{repair\_session\_max\_tree\_depth}},
{{scripted\_user\_defined\_functions\_enabled}},
{{use\_deterministic\_table\_id}}, {{cms\_default\_retry\_backoff}},
{{cms\_default\_max\_retry\_backoff}}. These are all {{@Deprecated}} in
Config.java. The test already skips them automatically, but should they also be
annotated {{@HiddenInYaml}} for clarity?
- \*Paxos v2 (16 fields)\*: All the {{paxos\_\*}} and
{{skip\_paxos\_repair\_\*}} fields from CEP-14. Internal LWT protocol tuning.
- \*TCM/CMS (12 fields)\*: CMS timeouts/retries ({{cms\_\*}}), metadata
snapshotting, discovery timeout, {{unsafe\_tcm\_mode}}, progress barrier fields
({{progress\_barrier\_\*}}), and {{short\_rpc\_timeout}}. All internal CEP-21
infrastructure.
- \*Accord (3 fields)\*: {{accord\_preaccept\_timeout}},
{{concurrent\_accord\_operations}}, {{consensus\_migration\_cache\_size}}.
CEP-15 internals, still maturing.
Does anyone see a reason any of these should be exposed in cassandra.yaml
rather than marked {{@HiddenInYaml}}? If we can agree on these groups, it
reduces the remaining discussion to about 13 individual fields where there are
differing opinions.
> Add a check if properties defined in Config.java but not mentioned in
> cassandra.yaml
> ------------------------------------------------------------------------------------
>
> Key: CASSANDRA-20249
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20249
> Project: Apache Cassandra
> Issue Type: Task
> Components: Build
> Reporter: Dmitry Konstantinov
> Assignee: Pedro Gordo
> Priority: Normal
> Fix For: 6.x
>
> Attachments: config_discrepancy.txt
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Triggered by https://lists.apache.org/thread/ror3ynqthssf4yghq3o2hlbd9jyx8y0l
> It is quite frequent case when a new property is added to Config.java but not
> mentioned in cassandra.yaml and cassandra_latest.yaml. Sometimes, it is made
> intentionally for some non-public parameters.
> We need to find a way to detect such discrepancies between Config.java vs
> cassandra.yaml and cassandra_latest.yaml automatically.
> Intentionally hidden properties can be marked with some Java annotation in
> Config.java such as @Hidden
> A sketch of the check idea:
> {code}
> String configUrl = "file:///..cassandra-trunk/conf/cassandra.yaml";
> Field[] allFields = Config.class.getFields();
> List<String> topLevelPropertyNames = new ArrayList<>();
> for(Field field : allFields)
> {
> if (!Modifier.isStatic(field.getModifiers()))
> {
> topLevelPropertyNames.add(field.getName());
> }
> }
> URL url = new URL(configUrl);
> List<String> lines = Files.readAllLines(Paths.get(url.toURI()));
> int missedCount = 0;
> for (String propertyName : topLevelPropertyNames)
> {
> boolean found = false;
> for (String line : lines)
> {
> if (line.startsWith(propertyName + ":")
> || line.startsWith("#" + propertyName + ":")
> || line.startsWith("# " + propertyName + ":")) { //TODO: replace
> with a regexp to allow any number of spaces..
> found = true;
> break;
> }
> }
> if (!found)
> {
> missedCount++;
> System.out.println(propertyName);
> }
> }
> System.out.println("Total missed:" + missedCount);
> {code}
> Output example: [^config_discrepancy.txt]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
