[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17125553#comment-17125553 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - [ [Part 2 |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3] ] [ [DTests |https://github.com/ekaterinadimitrova2/cassandra-dtest/tree/CASSANDRA-15234-3] ] [ [CCM |https://github.com/ekaterinadimitrova2/ccm/tree/CASSANDRA-15234-3] ] [ [CI run |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/187/workflows/615cdebf-1d49-47ab-93a4-efad25a247b1] ] - Unit tests updated - Getters and Setters names in the DatabaseDescriptor were updated to follow the new names of the parameters updated - The conversion of some of the parameters values which is happening in their getters is kept as it is needed in runtime. Some parameters are used with different units in the different areas of the codebase. So I left this part untouched. - New class created, ParseAndConvertUnits - it pulls all the functionality needed to cover the addition of units suffixes to the Memory, Duration, and Rate parameters in the cassandra.yaml - the cassandra.yaml file in this branch contains currently the new parameters with new names and unit suffixes but the arrangement of the sections is the old one as this was causing me issues every time I rebase/commit. If we confirm the format, this will be the last step to switch to the new format - Added some additional parsing and supporting conversion functions for the double duration parameters. - The 2 failing cqlsh tests were fixed locally and tested. It was formatting issue. -1 unit test failed - it doesn't look a related issue but will check the config loaded to ensure that tomorrow - 8 failing dtest, it doesn't look related but I am gonna compare tomorrow morning - Current version of the code needs a final rebase/merge of the last 6 commits on trunk. Will do tomorrow after I recheck the CI and current version - Have to return the default requirements.txt and config.yml files. The current versions were required only for testing purposes with CircleCI *WARNING:* we should be careful with the dtests and ccm. One missed check for old/new parameter and default value of the respective parameter from Config will be loaded. This might not lead to failing test and thus silently cover the miss. *NOTE:* Custom Types, as they were suggested, don't really work with SnakeYAML, unfortunately. SnakeYAML supports nothing more than mapping between the yaml and the Config class. * _So something like that: Memory max_value = new Memory("5ms") wouldn't work. If there is no value in the yaml, the config default max_value will be parsed properly but if there is value assigned to max_value at the yaml, it won't be assigned properly in Config. Classes are used for nested parameters. Also, currently there won't be a big benefit of the custom types as the different parameters validations in the db descriptor are taking different approaches, there is no general framework/approach._ _-_ The last part was updated as per the recommendations. CI ran successfully. Need to rebase and recheck nothing that might need the attention of this patch has changed in the meanwhile. Three weeks passed since I took care of this one. [ [Part3 |https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] ] [ [CI JAVA8 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/146/workflows/fae5da98-acf5-4824-bf61-3ffab3d86cd6]] [ [CI Java11 |https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/146/workflows/21b088c4-8984-4b20-8302-2ff8adaf55f7] ] > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17124520#comment-17124520 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Thanks [~mck] and [~snazy], that helped, pip3 install was provided with --upgrade in the circleci config file. Almost done, have to clean the code tomorrow morning. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123794#comment-17123794 ] Michael Semb Wever commented on CASSANDRA-15234: [~e.dimitrova], looks like you'll need the fix [~snazy] added [here|https://github.com/apache/cassandra-builds/commit/bf6bc98085330172f323c27b21810ac0eb0cc732]. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123750#comment-17123750 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Thanks [~mck], I will do the testing locally then and attach the logs here. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17123580#comment-17123580 ] Michael Semb Wever commented on CASSANDRA-15234: bq. Michael Semb Wever, is there a way this to be done in Jenkins? Currently the CCM repository is not a parameter to the Cassandra-devbranch pipeline. So I can only recommend the same approach, by updating the repo url in the cassandra-dtests/requirements.txt and starting the devbranch pipeline off against your patched dtest repo. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17121528#comment-17121528 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - I think that with my CCM patch all dtests should be fine now but I didn't manage to run them with my CCM patch in CircleCI. I was advised to change only the repo in requirements.txt but configuration errors appear and I can't run the tests. [~mck], is there a way this to be done in Jenkins? Otherwise I can try locally but this will take at least half a day to run the dtests... [~dcapwell] [~mck] if you have any ideas how to deal with this task efficiently will be appreciated. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120382#comment-17120382 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - 99% done. I already took care of all the things requested in the initial review. A couple of things we agreed on offline. Spent some time on the testing as all Dtests and CCM required edits because of all the renaming of parameters. Will be done for final review Monday. (running some final tests on my end after edit) > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17120295#comment-17120295 ] Josh McKenzie commented on CASSANDRA-15234: --- Where do we sit with this? One of our few beta 1 blockers. Thanks! > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-alpha > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094815#comment-17094815 ] David Capwell commented on CASSANDRA-15234: --- I am ok with enums > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094794#comment-17094794 ] Michael Semb Wever commented on CASSANDRA-15234: bq. Part3 - anyone any thoughts whether it should be 1 class or maybe divided into 3 classes, as suggested by David Capwell? I'm ok with one class. But it would be nice to list the properties and environments used in enums, so it is quick to list/read what Cassandra accesses. For example {{SystemPropertiesTable.java}} should be not be defining its own enums nor the string literals to the properties and environments. Otherwise part 3 looks good. What test coverage is there (or will there be) here? > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17091157#comment-17091157 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Latest update: Part1 - format issues fixed - The isOldYAML logger info changed to warn, and it lists also the old properties used - The format of yaml props is noun_verb? native_transport props adjusted to follow this. Max sent to the back as in native_transport_threads_max. - MAP_OLD_TO_NEW_PARAMETERS is not static (so it is freed memory after startup) Part2 - will be completed in the suggested way probably tomorrow (working asynchronously) Part3 - anyone any thoughts whether it should be 1 class or maybe divided into 3 classes as suggested by [~dcapwell]? > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089921#comment-17089921 ] Michael Semb Wever commented on CASSANDRA-15234: After a chat with [~e.dimitrova] I understand the null issue a bit better now. In the current codebase the {{int}} properties can not be left blank, but it can be commented out. - A commented value means a default of {{0}} (though it will also report an error in system.log). - A blank {{int}} property throws an exception. In this patch some of these {{int}} properties will become {{String}} properties. For example {{commitlog_sync_period_in_ms}} will become {{commitlog_sync_period}}, changing its type from {{int}} to {{String}}. The values will then permit unit suffixes, eg {{100ms}} and {{3s}}. These values still need to support the existing behaviour: blank throws an exception and commented out means zero. This is not possible with snakeyaml as, for {{String}} properties, both commented out and blank get interpreted as {{null}}. A suggestion to differentiate between a commented out {{String}} yaml property versus a blank {{String}} yaml property was to add a simple {{isPropertyBlank(name)}} method that performs a regular expression match against the plaintext of the yaml file. It is simple because the parsing of units suffix and {{int}} prefix happens in the C* code. The regexp match would then be: {{"^" + property + " *: *$"}} > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089439#comment-17089439 ] Michael Semb Wever commented on CASSANDRA-15234: bq. Part2 - null is a legit issue and no one of us likes the idea of adding "" in the yaml file. Still have to spend time to think about a workaround and that we already have it there. Some properties are nullable numbers, ie null is be a valid value. I would rather not permit/encourage the use of magic numbers (eg -1, or MIN_VALUE or MAX_VALUE) to notate the value as being undefined. An example is {{allocate_tokens_for_local_replication_factor}}. Its use as a nullable is in [BootStrapper.getBootstrapTokens(..)|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/dht/BootStrapper.java#L157-L179] Is this problem restricted to the parsing of deprecated (old yaml) properties? How many of the properties that are getting renamed were nullables? > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089156#comment-17089156 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - @David, thanks for the review and the clarifications and Slack constructive discussion! So looks like we have the same vision and I missed one thing. Let me wrap up here for the record and [~mck] who is also looking into it. Part2 - null is a legit issue and no one of us likes the idea of adding "" in the yaml file. Still have to spend time to think about a workaround and that we already have it there. The Memory class is kind of what I meant when I said I have to pull the current implementation separately. So the suggestion is to pull into three additional classes - Memory, Throughput and TimeDuration (or something similar) The confusion was about a thing I saw and forgot to take care. :\ There are 11 getters in the DatabaseDescriptor. They need to be changed to only getters without doing one more layer of conversion to bytes. Example: getMaxValueSizeInBytes() { return maxValue.toBytes(); } -> Memory getMaxValueSize() { return maxValue} This will clean the code and make it easy for maintenance. Part 3 - code-wise not related to Part1 and 2, just additional small task which was requested in the previous comments, considering we are trying to do standardization as part of this ticket. Whether we keep all of them in one class or we split is a matter of agreement I think. As soon as we get it, I will correct it. This is real quick. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089105#comment-17089105 ] David Capwell commented on CASSANDRA-15234: --- Part 3 appears to be a mix of com.google.common.base.StandardSystemProperty, JMX properties, and 3 Cassandra properties (focus on starting up; I know this is showing the idea, so totally fine with this); how I read the class is as the DatabaseDescriptor for system properties/env, is this the thinking? I kinda feel that it would be best to split this in 3 1) guava's StandardSystemProperty, if there is anything missing, see #2 2) JavaProperties (not tied to a name, just a placeholder). This stores the JMX and anything not in StandardSystemProperty 3) SysProperties. This manages Cassandra specific configs. Im not tied to that so flexible if people only want one class to rule them all... For part 3 I am more curious on how this ties into part 1 and 2; prototype or small informal writeup (1-2 sentences, just want to know your thinking) would be good. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17089094#comment-17089094 ] David Capwell commented on CASSANDRA-15234: --- Most of my experience with SnakeYaml is as a plugin to Jackson, normally Jackson is better at converting JSON/YAML -> Object and offers more flexibility; I may have misread you but the "int" null case isn't normally an issue in Jackson (assuming you define defaults). If this is a real limitation, wonder if it's better to use Jackson as the interface with SnakeYaml as the parser? I skimmed part 2 quickly. What I see is you offer two fields: string, and parsed value. If the user provides the string version we attempt to parse it and update the parsed value. In order to add a new field we would need to define the string and parsed versions, and provide the mapping to the MEM_UNITS_MAP map; my first thought is "why not make this a type?" I hope this example is clear to my thinking {code} class Memory { long bytes; public Memory(String value) { this.bytes = parse(value); } public long getBytes() { return bytes}; public long getKillaBytes() { return bytes * 1024 }; ... } {code} {code} Memory max_value_size = Memory.ofMb(256); {code} I quickly prototyped this for the field I showed and it parses just fine, also has the benefit that we drop a lot of unit conversions since config spec != runtime value (config is mb, runtime is bytes). My thinking is having types rather than the primitives is easier to read, easier to work with, easier to use with properties, etc; it also means we don't need to maintain the mapping from string to parsed fields. The other benefit is it plays nicely with part 1. We only have a single field, and if the user defines the new and the old the logic for which is used is centralized in a single place. Ill start reading part 3 now. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17086118#comment-17086118 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Done but with one caveat which was caught thankfully with the DTests. Unit tests were even on 100% green at some runs. (And my unit test was not covering this conversion, I should add it!) I am new to snakeyaml so I might be wrong, I would appreciate if someone who has the experience proves me wrong. >From the DTest failures I realized that turning into Strings the values from >the YAML we hit one issue. To me it looks like we already have it actually >with Integer and String parameters, just they are not many so it didn't byte >us up to now. Not sure whether this was considered when the Config class was >built or not. So the issue with int parameters in Config in my solution is: Currently in trunk, when we parse to int variable, If a parameter is commented in the cassandra.yaml, this is observed as null and it will raise an exception in some cases. (i.e. commitlog_sync_period_in_ms) But if it is not commented, just not assigned a value, then this is considered as 0 (as we parse to int). The situation is different with String, Integer. They will be in both cases null. No differentiator. But we don't have many parameters being Integer and String and it didn't byte us. Now with the units attached to the parameters in the cassandra.yaml file, we need to parse strings with snakeyaml and then convert to the proper int values in Config. A way to cope with this issue is to make the users use "". In this way, if we have a parameter not commented, but not also having an assigned value in cassandra.yaml, we can identify it by checking a string for being empty. If the parameter is commented or not presented, it will be null. But this would be one more new thing for the users to consider. That's why I came back to snakeyaml again and tried to think of a different way utilizing it. Unfortunately, up to now I didn't find any different solution. Also, the links to the examples in the official snakeyaml documentation are broken. Not sure whether there they had some interesting stuff. Maybe something with customized constructor but also, I try to make as less as possible disruptive changes in the codebase in order not to mess up something with the Config which might be extremely unpleasant. I will be happy to hear an experienced opinion here. Does anyone know a better solution? Or are we ok to introduce "" in the yaml for empty parameters/Strings recognition? [Part1|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-2] (This part was already revised by [~dcapwell]David but I paste the explanation again here for completeness). - New cassandra.yaml ---> currently cassandra-new.yaml in the branch as I wanted to test the backward compatibility on the old version (will have to rename properly also these yaml files with version probably) reorganization of the file renames to make the names more consistent - Backward compatibility with the old names - my approach is: Cassandra comes with the new version of the yaml to ensure any new builds will be using it During load of the yaml file I check whether old names are identified(means someone upgraded to 4 and wants to still use the old yaml). If this is the case, we load the values from the old yaml to the new variables (the already renamed parameters). Also, there is a warning to the user that there is new cassandra.yaml version and he/she can consider an update. [Part2|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-3] - units added to the values in the cassandra.yaml file. Parsing and unit conversion in place. One unit test added, should be further developed. [DTests|https://github.com/ekaterinadimitrova2/cassandra-dtest] updated accordingly for v.4 [Part3|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-5] - SysProperties class, accessors, hope I didn't miss anything... most of them are actually just using TO DO: Extract the parser/converter in a separate class from Config [~mck]? [~dcapwell]? [~benedict]? Anyone else any thoughts here? > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as th
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17075947#comment-17075947 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Part 1 was fixed and CI run showed no issues, forgot to update the ticket. Part II(capability for the users to pickup units) and Part III(the system properties accessors) will have to pass CI run tonight. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066778#comment-17066778 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - The branch is not compiling after I made some changes by mistake. I'll fix this later > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066295#comment-17066295 ] David Capwell commented on CASSANDRA-15234: --- bq. The old one won't be there anymore. I can leave it only in test/conf for test purposes. sounds good to me. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066285#comment-17066285 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - And I forgotyes, nested parameter backward compatibility does not exist in the current change as we don't have it. But I think (speculating as I didn't do it but it looks like that) should be easy to be added in case we need it at some point. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066284#comment-17066284 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - I mentioned in the initial submission... I propose the new version. Cassandra should come only with one version, the new one. But the old one was kept for now for the backward compatibilities CI run and verification. The idea is to have the new one v2. The old one won't be there anymore. I can leave it only in test/conf for test purposes. So that's why I didn't consider just to list all the parameters and say rename so they rename them in the old version. v2 logic I mentioned before that I am gonna add it and no one objected. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066269#comment-17066269 ] David Capwell commented on CASSANDRA-15234: --- bq. I agree with this. On the other hand, we keep all max/min/etc at the end/start of a name, that was the initial plan I think. naming is hard, ill defer to others who have stronger opinions bq. they will be a lot. yep, which is why it would be good to tell the users which ones are deprecated and replaced with what. If we just say "there is a problem" they will find out what the problem was in 5.0 (and only then try to figure out how to migrate). bq. Also, my point was that we change completely the version of the yaml. I saw you have v2 logic, but don't understand why given the current patch; is there more pending which change the structure? bq. Did you check it how it looks like now? Do you mean the YAML? the log looks the [same|https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-2#diff-5749c5e00813f3ae16b68ff8de095cc0R305] to me. I see you have the two yaml; not sure about others but IMO we should only have 1 file (since users see it), two could be confusing. bq. We don't have that issue now but it looks like snakeyaml has everything in place in order to deal with nested parameters, too. Not sure I responded to your question. the current patch doesn't have the need, was just thinking about if it came up and the impact to the rename logic; the patch currently makes sense for top level names. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17066238#comment-17066238 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Hi [~dcapwell] Thanks a lot for your feedback! "https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-2#diff-b66584c9ce7b64019b5db5a531deeda1R189 I prefer native_transport_max_threads over max_native_transport_threads, the native_transport prefix scopes things better IMO and matches the other configs as well (such as native_transport_flush_in_batches_legacy)." I agree with this. On the other hand, we keep all max/min/etc at the end/start of a name, that was the initial plan I think. "log isn't actionable, personally feel logging which properties were used but deprecated is easier to act on (so may be better to log per property?)" I was thinking about this, too. But in this case we don't change only one or two parameters but 15 for now. And when I also add the change of those which had a unit at the back... they will be a lot. Also, my point was that we change completely the version of the yaml. Did you check it how it looks like now? So from now on any changes/comments/recommendations will go there. We don't want people to start making changes on the old yaml but potentially to update it with the new one. "For the parsing remapping logic, I am good with this, it keeps Config.java clean and removes the need for DatabaseDescriptor to do the remapping. Yes, that was exactly my point. Also, we can easily get rid of this backward compatibility one day in the future if we want to. My main question is how this works with a nested property? Lets say we want to change the yaml to seed_provider: # renamed class_name to name - name: org.apache.cassandra.locator.SimpleSeedProvider parameters: - seeds: "127.0.0.1:7000" Not sure if we care about something like that now, but its the main question I have on the remapping (and potential conflict where top level != nested)." We don't have that issue now but it looks like snakeyaml has everything in place in order to deal with nested parameters, too. Not sure I responded to your question. "I personally prefer the parsing in the file loading since it can give more meaningful feedback to users than failing somewhere else. Also would mean that Config.java always has the final results" Same here, thanks. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065977#comment-17065977 ] David Capwell commented on CASSANDRA-15234: --- https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-2#diff-b66584c9ce7b64019b5db5a531deeda1R189 I prefer native_transport_max_threads over max_native_transport_threads, the native_transport prefix scopes things better IMO and matches the other configs as well (such as native_transport_flush_in_batches_legacy). https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-2#diff-5749c5e00813f3ae16b68ff8de095cc0R238 I feel it makes sense to put that in the if block https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-2#diff-5749c5e00813f3ae16b68ff8de095cc0R227 and remove isOldYAML https://github.com/apache/cassandra/compare/trunk...ekaterinadimitrova2:CASSANDRA-15234-2#diff-5749c5e00813f3ae16b68ff8de095cc0R305 log isn't actionable, personally feel logging which properties were used but deprecated is easier to act on (so may be better to log per property?) For the parsing remapping logic, I am good with this, it keeps Config.java clean and removes the need for DatabaseDescriptor to do the remapping. My main question is how this works with a nested property? Lets say we want to change the yaml to {code} seed_provider: # renamed class_name to name - name: org.apache.cassandra.locator.SimpleSeedProvider parameters: - seeds: "127.0.0.1:7000" {code} Not sure if we care about something like that now, but its the main question I have on the remapping (and potential conflict where top level != nested). bq. For example, still wondering a bit about the best place to incorporate the parsing(also the unit converts...) to ensure least disruptions in the current code logic. Looks to me that it might be good again to call it during the yaml file load. I personally prefer the parsing in the file loading since it can give more meaningful feedback to users than failing somewhere else. Also would mean that Config.java always has the final results > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17065133#comment-17065133 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - [Part1|https://github.com/ekaterinadimitrova2/cassandra/tree/CASSANDRA-15234-2] - dtests are still running and have to clear/correct some comments in config but all in-jvm and unit tests look fine to me. - New cassandra.yaml ---> currently cassandra-new.yaml in the branch # reorganization of the file # renames to make the names more consistent - Backward compatibility with the old names - my approach is: # Cassandra comes with the new version of the yaml to ensure any new builds will be using it # During load of the yaml file I check whether old names are identified(means someone upgraded to 4 and wants to still use the old yaml). If this is the case, we load the values from the old yaml to the new variables (the already renamed parameters). Might need some refactoring here and there but I found this as the easiest and least disruptive approach. Also, there is a warning to the user that there is new yaml version and he/she can consider an update. Also TO DO for later: change cassandra.yaml in conf and test/conf and put cassandra-old.yaml in the test/confShould update also CHANGES.txt and NEWS.txt at some point when I am done... Still have to complete some parts of the other stuff before sharing. For example, still wondering a bit about the best place to incorporate the parsing to ensure least disruptions in the current code logic. Looks to me that it might be good again to call it during the yaml file load. Should think about it a bit more. Any constructive feedback is always appreciated. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061075#comment-17061075 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Thanks [~benedict] for the confirmation on what you had in mind here. Makes sense to me. So just to confirm I am not doing something stupid. For the backward compatibility we are simply using either the old or new parameter from yaml in Config/SysProperties, depending on which one the user used there? In the Database Descriptor I will add a warning which will also warn the developers that they can already use the new one. How about that? Or something else is meant in this case for backward compatibility? I already created a class of accessors for the SystemProperties. Just want to think also maybe about some of the environment variables too. The config is reshuffled and many renames are already in place so I guess after this confirmation and a couple of other things to complete, I will submit soon the commits for review. Thanks > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058138#comment-17058138 ] David Capwell commented on CASSANDRA-15234: --- Yep, think we are on the same page here; I just prefer smaller patches so rather have ~3-4 (based off your split) patches rather than the same number of commits but one patch. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17058024#comment-17058024 ] Benedict Elliott Smith commented on CASSANDRA-15234: My personal view is that there's no need to overcomplicate this ticket, and that perhaps David Capwell anticipates greater complexity in addressing it than I do. My goal is to make the config more intuitive, consistent and also to not make any assumptions about reasonable units. Every property has its own units picked arbitrarily, and this is confusing. Publishing a simple regex to validate the value types for tools seems sufficient to address [~rssvihla]'s concerns, I think? And I don't think we should make it pluggable, just accept quantities and rates, and we can be quite restrictive, even just accepting GiB/s, MiB/s and KiB/s. My personal approach would be to, in separate commits but one ticket # Move properties to a single file, with strongly typed methods and sensible names for fetching the property. # Note property names present in either {{Config}} or via system properties, that relate to the same concept but use different terms - not just prefixes like otc_, but also: #* Why do we use {{enable}}, {{enabled}} and {{disable}} in our property names? Why does it sometimes go at the start or end? #* Why do we use both {{dc}} and {{datacenter}}? #* Who does {{max}} sometimes go at the start, sometimes in the middle? #* ... # Rename things, and make sure {{Config}} and system properties look for both old and new forms # Done > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057969#comment-17057969 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Ok, back to this one. Unfortunately, I was parallelizing with some other stuff at the beginning of the week. Thanks [~mck] for sharing that ticket, that was helpful. I started from looking into standardizing the parameters names and 1) of the plan [~dcapwell] shared. Also, my idea was to reshuffle a bit the order of the parameters in the yaml and try to put them in sections, for example: * Quick start The minimal properties needed for configuring a cluster. * Commonly used Properties most frequently used when configuring Cassandra. * Performance Tuning Tuning performance and system resource utilization, including commit log, compaction, memory, disk I/O, CPU, reads, and writes. * Advanced Properties for advanced users or properties that are less commonly used. * SecurityServer and client security settings. Yes, this idea came after looking at online documentations and trying to think from user's perspective. On another topic, looking at the plan, I realized that you might want actually a parser and unit converter for the values in order to give freedom to the users to add whatever value they would like in the yaml. Initially, I was thinking that we talk about the suffixes of the parameters names, the way we see them now and some name changes and backward compatibilities. So my question is what is the reason behind doing this and shall we split this Jira then in a couple of smaller ones if we go into that direction? > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17057944#comment-17057944 ] Ryan Svihla commented on CASSANDRA-15234: - I think this is overall a good idea, but it definitely needs some limits on the number of combinations supported. Let's just look at a hypothetical new configuration `compaction_throughput` instead of `compaction_throughput_mb_per_sec` (which is a poster child for a difficult to remember name in need of review): * Any downstream tooling that reads configuration will have to take on everything we add, which is fine, but the more math we require the worse it is on them to get updated. An older tool will read compaction_throughput_mb_per_sec and it was self documenting. A new tool will have to take into account every variant we support for MiB/MB/mb/bytes/etc/etc * what about `500mb` vs `500mb/s` vs `500 mbs` (note the space) vs `500MiB/s` vs `500 mb/s` (note the 2 spaces). Which of those is obviously valid or wrong at a glance to a new user? So if we're going to do it, definitely only accept one valid that's the same for everything..I still think it's adding some learning curve for new users. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050573#comment-17050573 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - Great, then I am gonna start working on it tomorrow! Probably will ask some questions at some point. Thanks! > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict Elliott Smith >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050544#comment-17050544 ] Michael Semb Wever commented on CASSANDRA-15234: And there are a lot of accessing jvm args using the "cassandra." string literal prefix, instead of {{Config.PROPERTY_PREFIX}} This ticket would help simplify {{SystemPropertiesTable. CASSANDRA_RELEVANT_PROPERTIES}} in CASSANDRA-15616 > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050541#comment-17050541 ] David Capwell commented on CASSANDRA-15234: --- Nope. All yours if you want > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17050476#comment-17050476 ] Ekaterina Dimitrova commented on CASSANDRA-15234: - [~dcapwell]. did you start this one? > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17026288#comment-17026288 ] David Capwell commented on CASSANDRA-15234: --- I see a few things in this JIRA 1) centralize location of JVM args 2) backwards compatible way to rename properties (JVM args and yaml) 3) support common unit conversions (time, space, etc.) 4) type safe values (Rate, Duration, Space, etc) 5) backwards compatible value parsing 6) document all the things I should be able to start taking on parts of this in a few weeks, unless anyone else wants to in the mean time. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004571#comment-17004571 ] Benedict Elliott Smith commented on CASSANDRA-15234: Yep. Hopefully I'd have noticed when implementing, but thanks for making sure! > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > ms|millis(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17004567#comment-17004567 ] Jon Haddad commented on CASSANDRA-15234: Probably want ms (milliseconds) in there as well. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict Elliott Smith >Priority: Normal > Fix For: 4.0, 4.0-beta > > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886938#comment-16886938 ] Benedict commented on CASSANDRA-15234: -- Definitely. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Priority: Normal > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15234) Standardise config and JVM parameters
[ https://issues.apache.org/jira/browse/CASSANDRA-15234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886937#comment-16886937 ] Aleksey Yeschenko commented on CASSANDRA-15234: --- Would be nice, while at it, to pull all JVM args into one class, a-la {{Config}}, with strongly typed accessors and proper documentation for each arg. Right now you have to look for usage of {{System.getProperty()}}, {{Boolean.getBoolean()}}, etc., to enumerate them all, which is a mess. > Standardise config and JVM parameters > - > > Key: CASSANDRA-15234 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15234 > Project: Cassandra > Issue Type: Bug >Reporter: Benedict >Priority: Normal > > We have a bunch of inconsistent names and config patterns in the codebase, > both from the yams and JVM properties. It would be nice to standardise the > naming (such as otc_ vs internode_) as well as the provision of values with > units - while maintaining perpetual backwards compatibility with the old > parameter names, of course. > For temporal units, I would propose parsing strings with suffixes of: > {{code}} > u|micros(econds?)? > s(econds?)? > m(inutes?)? > h(ours?)? > d(ays?)? > mo(nths?)? > {{code}} > For rate units, I would propose parsing any of the standard {{B/s, KiB/s, > MiB/s, GiB/s, TiB/s}}. > Perhaps for avoiding ambiguity we could not accept bauds {{bs, Mbps}} or > powers of 1000 such as {{KB/s}}, given these are regularly used for either > their old or new definition e.g. {{KiB/s}}, or we could support them and > simply log the value in bytes/s. -- This message was sent by Atlassian JIRA (v7.6.14#76016) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org