[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17573025#comment-17573025 ] Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/29/22 3:16 PM: -- Thank you. Wrapped up [4.1|https://github.com/ekaterinadimitrova2/cassandra/commit/3b8cefe5136937dad8e9188216eefd70a15142a4] and [trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/6e9eb2a4c92b6a77dd3051d4782f1f21fc08ae18] in new branches and kept the old PRs unsquashed for reference. CI results pending. 4.1 CI run started [here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=17737-4.1-part2], trunk CI pushed [here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=17737-trunk-part2] For the record - this commit intends to fix a null bug we identified in Converters and the disabled value for sstable_preemptive_open_interval_in_mb. In order to start using @Nullable annotation for Config properties, I had to change com.google.code.findbugs scope from provided to default compile one. Looking into its license [https://opensource.org/licenses/BSD-3-Clause] and reading here - [https://www.apache.org/legal/resolved.html] we agreed the change is acceptable. Not brought to the mailing list as we are not adding brand new dependency. Also, we agreed to open a follow-up improvement ticket so that all nullable Config properties start using the @Nullable annotation - CASSANDRA-17785 was (Author: e.dimitrova): Thank you. Wrapped up [4.1|https://github.com/ekaterinadimitrova2/cassandra/commit/3b8cefe5136937dad8e9188216eefd70a15142a4] and [trunk|https://github.com/ekaterinadimitrova2/cassandra/commit/6e9eb2a4c92b6a77dd3051d4782f1f21fc08ae18] in new branches and kept the old PRs unsquashed for reference. CI results pending. 4.1 CI run started [here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=17737-4.1-part2], trunk CI pushed [here|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=17737-trunk-part2] For the record - this commit intends to fix a null bug we identified in Converters and the disabled value for sstable_preemptive_open_interval_in_mb. In order to start using @Nullable annotation for Config properties, I had to change com.google.code.findbugs scope from provided to default compile one. Looking into its license [https://opensource.org/licenses/BSD-3-Clause] and reading here - [https://www.apache.org/legal/resolved.html] we agreed the change is acceptable. Not brought to the mailing list as we are not adding brand new dependency. > Validate that JMX updates properly any properties that were moved to the new > config classes > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval, user_defined_functions_fail_timeout, > write_request_timeout, cdc_free_space_check_interval, roles_validity, > internode_streaming_tcp_user_timeout, gc_warn_threshold, > range_request_timeout, credentials_update_interval, truncate_request_timeout, > cas_contention_timeout, periodic_commitlog_sync_lag_block, > streaming_state_expires, repair_request_timeout, permissions_update_interval, > dynamic_snitch_reset_interval, internode_tcp_connect_timeout, > paxos_purge_grace_period, dynamic_snitch_update_interval, > trace_type_query_ttl, denylist_initial_load_retry,
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572622#comment-17572622 ] Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/28/22 7:35 PM: -- I rebased and reran CI, it seems there was only one known failure on trunk - CASSANDRA-17618. Starting commit of the fix for the following issues: * Fixed non-mutable prepared_statements_cache_size_mb, key_cache_size_in_mb, counter_cache_size_in_mb to be updated in Settings Virtual Table on startup * Fixed conf.index_summary_capacity_in_mb * Fixed two GCInspector properties I will add a table to the ticket description with all commits that will be done as part of this ticket to make things organized. was (Author: e.dimitrova): Rebased and rerun CI, it seems there was only one known failure on trunk - CASSANDRA-17618. Starting commit of the fix for the following issues: * Fixed non-mutable prepared_statements_cache_size_mb, key_cache_size_in_mb, counter_cache_size_in_mb to be updated in Settings Virtual Table on startup * Fixed conf.index_summary_capacity_in_mb * Fixed two GCInspector properties I will add a table to the ticket description with all commits that will be done as part of this ticket to make things organized. > Validate that JMX updates properly any properties that were moved to the new > config classes > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval, user_defined_functions_fail_timeout, > write_request_timeout, cdc_free_space_check_interval, roles_validity, > internode_streaming_tcp_user_timeout, gc_warn_threshold, > range_request_timeout, credentials_update_interval, truncate_request_timeout, > cas_contention_timeout, periodic_commitlog_sync_lag_block, > streaming_state_expires, repair_request_timeout, permissions_update_interval, > dynamic_snitch_reset_interval, internode_tcp_connect_timeout, > paxos_purge_grace_period, dynamic_snitch_update_interval, > trace_type_query_ttl, denylist_initial_load_retry, commitlog_sync_period, > native_transport_idle_timeout, credentials_validity, > validation_preview_purge_head_start, repair_state_expires, > internode_tcp_user_timeout, trace_type_repair_ttl, cache_load_timeout, > commitlog_sync_group_window, slow_query_log_timeout, > counter_write_request_timeout, user_defined_functions_warn_timeout > *DataStorageSpec* - > internode_application_send_queue_reserve_endpoint_capacity, cdc_total_space, > networking_cache_size, commitlog_total_space, > internode_application_send_queue_capacity, key_cache_size, > memtable_heap_space, trickle_fsync_interval, max_hints_size_per_host, > internode_application_receive_queue_reserve_endpoint_capacity, > native_transport_max_frame_size, coordinator_read_size_warn_threshold , > internode_application_receive_queue_reserve_global_capacity, > internode_max_message_size, file_cache_size, local_read_size_fail_threshold, > data_disk_usage_max_disk_size, memtable_offheap_space, > coordinator_read_size_fail_threshold, counter_cache_size, > prepared_statements_cache_size, batchlog_replay_throttle, > row_index_read_size_fail_threshold, index_summary_capacity, > repair_session_space, paxos_cache_size, collection_size_fail_threshold, > internode_application_send_queue_reserve_global_capacity, column_index_size, > native_transport_receive_queue_capacity, sstable_preemptive_open_interval, >
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17572216#comment-17572216 ] Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/28/22 3:43 AM: -- I just pushed one more commit, suggested by [~dcapwell] using the @Nullable annotation, that solves the null issue on startup and in fromMap method. I totally forgot about that annotation that we can actually use it for this here, similar to what was done with @Deprecated. Many thanks for reminding me! was (Author: e.dimitrova): I just pushed one more commit, suggested by [~dcapwell] using the @Nullable annotation, that solves the null issue on startup and in fromMap method. I totally forgot about that annotation that we can actually use it for here. Many thanks for reminding me! > Validate that JMX updates properly any properties that were moved to the new > config classes > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval, user_defined_functions_fail_timeout, > write_request_timeout, cdc_free_space_check_interval, roles_validity, > internode_streaming_tcp_user_timeout, gc_warn_threshold, > range_request_timeout, credentials_update_interval, truncate_request_timeout, > cas_contention_timeout, periodic_commitlog_sync_lag_block, > streaming_state_expires, repair_request_timeout, permissions_update_interval, > dynamic_snitch_reset_interval, internode_tcp_connect_timeout, > paxos_purge_grace_period, dynamic_snitch_update_interval, > trace_type_query_ttl, denylist_initial_load_retry, commitlog_sync_period, > native_transport_idle_timeout, credentials_validity, > validation_preview_purge_head_start, repair_state_expires, > internode_tcp_user_timeout, trace_type_repair_ttl, cache_load_timeout, > commitlog_sync_group_window, slow_query_log_timeout, > counter_write_request_timeout, user_defined_functions_warn_timeout > *DataStorageSpec* - > internode_application_send_queue_reserve_endpoint_capacity, cdc_total_space, > networking_cache_size, commitlog_total_space, > internode_application_send_queue_capacity, key_cache_size, > memtable_heap_space, trickle_fsync_interval, max_hints_size_per_host, > internode_application_receive_queue_reserve_endpoint_capacity, > native_transport_max_frame_size, coordinator_read_size_warn_threshold , > internode_application_receive_queue_reserve_global_capacity, > internode_max_message_size, file_cache_size, local_read_size_fail_threshold, > data_disk_usage_max_disk_size, memtable_offheap_space, > coordinator_read_size_fail_threshold, counter_cache_size, > prepared_statements_cache_size, batchlog_replay_throttle, > row_index_read_size_fail_threshold, index_summary_capacity, > repair_session_space, paxos_cache_size, collection_size_fail_threshold, > internode_application_send_queue_reserve_global_capacity, column_index_size, > native_transport_receive_queue_capacity, sstable_preemptive_open_interval, > max_mutation_size, min_free_space_per_drive, batch_size_fail_threshold, > hinted_handoff_throttle, row_index_read_size_warn_threshold, max_value_size, > column_index_cache_size, compaction_large_partition_warning_threshold, > max_hints_file_size, collection_size_warn_threshold, > native_transport_max_request_data_in_flight, > internode_socket_receive_buffer_size, > internode_application_receive_queue_capacity, > internode_socket_send_buffer_size,
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571650#comment-17571650 ] Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/27/22 1:15 PM: -- So the property I am wondering about is sstable_preemptive_open_interval. It seems we have a [check|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/SSTableRewriter.java#L109] for being less than 0, I can see that for Windows we were setting it to -1 [here|https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05#diff-054af65b8d690b0fddc3e0a4ef05a80d8f1d6689b4f77912795fec019200666cL2806] before the windows support was removed, not anymore. Negatives are not documented anywhere and in 4.1 we cannot set it to negatives anymore (after CASSANDRA-15234). Not sure whether we should add a converter similar to other properties that if this one was set to negative value with the old name sstable_preemptive_open_interval_in_mb we should assign null with the new name post 4.1(keep also the backward compatibility sstable_preemptive_open_interval_in_mb=-1 meaning disabled) and use that one. I suspect it was left undocumented and advanced option? Or was -1 only for Windows?? [~benedict] I see you added this one long time ago in CASSANDRA-6916, do you think you can advise me here, please? Or maybe [~stefan.miklosovic] or [~jmckenzie] can advise me as you were working on removing the Windows support classes in CASSANDRA-16956? Trying not to break users. was (Author: e.dimitrova): So the property I am wondering about is sstable_preemptive_open_interval. It seems we have a [check|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/SSTableRewriter.java#L109] for being less than 0, I can see that for Windows we were setting it to -1 [here|https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05#diff-054af65b8d690b0fddc3e0a4ef05a80d8f1d6689b4f77912795fec019200666cL2806] before the windows support was removed, not anymore. Negatives are not documented anywhere and in 4.1 we cannot set it to negatives anymore (after CASSANDRA-15234). Not sure whether we should add a converter similar to other properties that if this one was set to negative value with the old name sstable_preemptive_open_interval_in_mb we should assign null with the new name post 4.1 and use that one. I suspect it was left undocumented and advanced option? Or was -1 only for Windows?? [~benedict] I see you added this one long time ago in CASSANDRA-6916, do you think you can advise me here, please? Or maybe [~stefan.miklosovic] or [~jmckenzie] can advise me as you were working on removing the Windows support classes in CASSANDRA-16956? Trying not to break users. > Validate that JMX updates properly any properties that were moved to the new > config classes > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval, user_defined_functions_fail_timeout, > write_request_timeout, cdc_free_space_check_interval, roles_validity, > internode_streaming_tcp_user_timeout, gc_warn_threshold, > range_request_timeout, credentials_update_interval, truncate_request_timeout, > cas_contention_timeout, periodic_commitlog_sync_lag_block, > streaming_state_expires, repair_request_timeout, permissions_update_interval, >
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[
https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571716#comment-17571716
]
Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/27/22 1:07 PM:
--
{quote}That said, I think we need to make sure we continue to respect the "-1
== disabled" aspect of this config since there's clusters out there in the wild
that are likely depending on that API contract.
{quote}
Absolutely! Now that we confirmed it I am going to push the patch as described
tomorrow morning. It’s a pity it was not documented, otherwise it would have
been already in place. But with the new framework it is a standard quick fix to
add a backward compatibility so not a problem, we will have it. Thank you both
for confirming it!
was (Author: e.dimitrova):
Bq. That said, I think we need to make sure we continue to respect the "-1 ==
disabled" aspect of this config since there's clusters out there in the wild
that are likely depending on that API contract.
Absolutely! Now that we confirmed it I am going to push the patch as described
tomorrow morning. It’s a pity it was not documented, otherwise it would have
been already in place. But with the new framework it is a standard quick fix to
add a backward compatibility so not a problem, we will have it. Thank you both
for confirming it!
> Validate that JMX updates properly any properties that were moved to the new
> config classes
> ---
>
> Key: CASSANDRA-17737
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17737
> Project: Cassandra
> Issue Type: Task
> Components: Local/Config
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x
>
>
> Check that any properties moved to the new types in 4.1 (Duration, Data
> Storage, Data Rate) are always updated by JMX and there are no inconsistent
> validations that might cover bugs. Validate proper update in Settings Virtual
> Table
> I branched the configCompatibilityTest in 4.1 in order to get the lists of
> properties in this
> [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137]
> - the commit won't get into the codebase, just pasting so people know how
> the lists were generated:
> *DataRateSpec* - entire_sstable_stream_throughput_outbound,
> inter_dc_stream_throughput_outbound,
> entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput,
> stream_throughput_outbound - we have setGet* tests and those seem solid, no
> issues found.
> *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh,
> request_timeout, hints_flush_period, read_request_timeout,
> index_summary_resize_interval, streaming_keep_alive_period, max_hint_window,
> roles_update_interval, user_defined_functions_fail_timeout,
> write_request_timeout, cdc_free_space_check_interval, roles_validity,
> internode_streaming_tcp_user_timeout, gc_warn_threshold,
> range_request_timeout, credentials_update_interval, truncate_request_timeout,
> cas_contention_timeout, periodic_commitlog_sync_lag_block,
> streaming_state_expires, repair_request_timeout, permissions_update_interval,
> dynamic_snitch_reset_interval, internode_tcp_connect_timeout,
> paxos_purge_grace_period, dynamic_snitch_update_interval,
> trace_type_query_ttl, denylist_initial_load_retry, commitlog_sync_period,
> native_transport_idle_timeout, credentials_validity,
> validation_preview_purge_head_start, repair_state_expires,
> internode_tcp_user_timeout, trace_type_repair_ttl, cache_load_timeout,
> commitlog_sync_group_window, slow_query_log_timeout,
> counter_write_request_timeout, user_defined_functions_warn_timeout
> *DataStorageSpec* -
> internode_application_send_queue_reserve_endpoint_capacity, cdc_total_space,
> networking_cache_size, commitlog_total_space,
> internode_application_send_queue_capacity, key_cache_size,
> memtable_heap_space, trickle_fsync_interval, max_hints_size_per_host,
> internode_application_receive_queue_reserve_endpoint_capacity,
> native_transport_max_frame_size, coordinator_read_size_warn_threshold ,
> internode_application_receive_queue_reserve_global_capacity,
> internode_max_message_size, file_cache_size, local_read_size_fail_threshold,
> data_disk_usage_max_disk_size, memtable_offheap_space,
> coordinator_read_size_fail_threshold, counter_cache_size,
> prepared_statements_cache_size, batchlog_replay_throttle,
> row_index_read_size_fail_threshold, index_summary_capacity,
> repair_session_space, paxos_cache_size, collection_size_fail_threshold,
> internode_application_send_queue_reserve_global_capacity, column_index_size,
>
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[
https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571665#comment-17571665
]
Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/27/22 12:29 AM:
---
Hey [~jonmeredith], thank you so much for the quick response. Appreciate it!
I am not sure what you mean by:
{quote}Invalid annotations, you have more than one @Replaces annotation in
Config class with same old name(sstable_preemptive_open_interval_in_mb)
defined.
{quote}
I guess you hit this when trying to add new property and using second
annotation with the same old name but for new property?
What we can do and we already do for other properties is change the Converter
in the @Replaces for sstable_preemptive_open_interval_in_mb
If we change the converter Converters.MEBIBYTES_DATA_STORAGE_INT for this
property with a new one - Converters.NEGATIVE_MEBIBYTES_DATA_STORAGE_INT, that
should do the magic
Any negative sstable_preemptive_open_interval_in_mb will be equal to setting
sstable_preemptive_open_interval as null 4.1+.
To see what I mean you might want to look at the Converters class,
NEGATIVE_SECONDS_DURATION is a similar converter but for duration properties.
I can create a patch tomorrow morning as part of this ticket
was (Author: e.dimitrova):
Hey [~jonmeredith], thank you so much for the quick response. Appreciate it!
I am not sure what you mean by:
{quote}Invalid annotations, you have more than one @Replaces annotation in
Config class with same old name(sstable_preemptive_open_interval_in_mb)
defined.
{quote}
I guess you hit this when trying to add new property and using second
annotation with the same old name but for new property?
What we can do and we already do for other properties is change the Converter
in the @Replaces for sstable_preemptive_open_interval_in_mb
If we change the converter Converters.MEBIBYTES_DATA_STORAGE_INT for this
property with a new one - Converters.NEGATIVE_MEBIBYTES_DATA_STORAGE_INT, that
should do the magic
any negative sstable_preemptive_open_interval_in_mb will be equal to setting
sstable_preemptive_open_interval as null 4.1+.
To see what I mean you might want to look at the Converters class,
NEGATIVE_SECONDS_DURATION is a similar converter but for duration properties.
I can create a patch tomorrow morning as part of this ticket
> Validate that JMX updates properly any properties that were moved to the new
> config classes
> ---
>
> Key: CASSANDRA-17737
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17737
> Project: Cassandra
> Issue Type: Task
> Components: Local/Config
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x
>
>
> Check that any properties moved to the new types in 4.1 (Duration, Data
> Storage, Data Rate) are always updated by JMX and there are no inconsistent
> validations that might cover bugs. Validate proper update in Settings Virtual
> Table
> I branched the configCompatibilityTest in 4.1 in order to get the lists of
> properties in this
> [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137]
> - the commit won't get into the codebase, just pasting so people know how
> the lists were generated:
> *DataRateSpec* - entire_sstable_stream_throughput_outbound,
> inter_dc_stream_throughput_outbound,
> entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput,
> stream_throughput_outbound - we have setGet* tests and those seem solid, no
> issues found.
> *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh,
> request_timeout, hints_flush_period, read_request_timeout,
> index_summary_resize_interval, streaming_keep_alive_period, max_hint_window,
> roles_update_interval, user_defined_functions_fail_timeout,
> write_request_timeout, cdc_free_space_check_interval, roles_validity,
> internode_streaming_tcp_user_timeout, gc_warn_threshold,
> range_request_timeout, credentials_update_interval, truncate_request_timeout,
> cas_contention_timeout, periodic_commitlog_sync_lag_block,
> streaming_state_expires, repair_request_timeout, permissions_update_interval,
> dynamic_snitch_reset_interval, internode_tcp_connect_timeout,
> paxos_purge_grace_period, dynamic_snitch_update_interval,
> trace_type_query_ttl, denylist_initial_load_retry, commitlog_sync_period,
> native_transport_idle_timeout, credentials_validity,
> validation_preview_purge_head_start, repair_state_expires,
> internode_tcp_user_timeout, trace_type_repair_ttl, cache_load_timeout,
> commitlog_sync_group_window, slow_query_log_timeout,
> counter_write_request_timeout,
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571650#comment-17571650 ] Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/26/22 11:05 PM: --- So the property I am wondering about is sstable_preemptive_open_interval. It seems we have a [check|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/SSTableRewriter.java#L109] for being less than 0, I can see that for Windows we were setting it to -1 [here|https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05#diff-054af65b8d690b0fddc3e0a4ef05a80d8f1d6689b4f77912795fec019200666cL2806] before the windows support was removed, not anymore. Negatives are not documented anywhere and in 4.1 we cannot set it to negatives anymore (after CASSANDRA-15234). Not sure whether we should add a converter similar to other properties that if this one was set to negative value with the old name sstable_preemptive_open_interval_in_mb we should assign null with the new name post 4.1 and use that one. I suspect it was left undocumented and advanced option? Or was -1 only for Windows?? [~benedict] I see you added this one long time ago in CASSANDRA-6916, do you think you can advise me here, please? Or maybe [~stefan.miklosovic] or [~jmckenzie] can advise me as you were working on removing the Windows support classes in CASSANDRA-16956? Trying not to break users. was (Author: e.dimitrova): So the property I am wondering about is sstable_preemptive_open_interval. It seems we have a [check|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/SSTableRewriter.java#L109] for being less than 0, I can see that for Windows we were setting it to -1 [here|https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05#diff-054af65b8d690b0fddc3e0a4ef05a80d8f1d6689b4f77912795fec019200666cL2806] before the windows support was removed, not anymore. Negatives are not documented anywhere and in 4.1 we cannot set it to negatives anymore (after CASSANDRA-15234). Not sure whether we should add a converter similar to other properties that if this one was set to negative value with the old name sstable_preemptive_open_interval_in_mb we should assign null with the new name post 4.1 and use that one. I suspect it was left undocumented and advanced option? Or was -1 only for Windows?? [~benedict] I see you added this one long time ago in CASSANDRA-6916, do you think you can advise me here, please? Or maybe [~stefan.miklosovic] , [~brandon.williams] or [~jmckenzie] can advise me as you were working on removing the Windows support classes? Trying not to break users. > Validate that JMX updates properly any properties that were moved to the new > config classes > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval, user_defined_functions_fail_timeout, > write_request_timeout, cdc_free_space_check_interval, roles_validity, > internode_streaming_tcp_user_timeout, gc_warn_threshold, > range_request_timeout, credentials_update_interval, truncate_request_timeout, > cas_contention_timeout, periodic_commitlog_sync_lag_block, > streaming_state_expires, repair_request_timeout, permissions_update_interval, > dynamic_snitch_reset_interval, internode_tcp_connect_timeout, > paxos_purge_grace_period,
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571650#comment-17571650 ] Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/26/22 11:04 PM: --- So the property I am wondering about is sstable_preemptive_open_interval. It seems we have a [check|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/SSTableRewriter.java#L109] for being less than 0, I can see that for Windows we were setting it to -1 [here|https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05#diff-054af65b8d690b0fddc3e0a4ef05a80d8f1d6689b4f77912795fec019200666cL2806] before the windows support was removed, not anymore. Negatives are not documented anywhere and in 4.1 we cannot set it to negatives anymore (after CASSANDRA-15234). Not sure whether we should add a converter similar to other properties that if this one was set to negative value with the old name sstable_preemptive_open_interval_in_mb we should assign null with the new name post 4.1 and use that one. I suspect it was left undocumented and advanced option? Or was -1 only for Windows?? [~benedict] I see you added this one long time ago in CASSANDRA-6916, do you think you can advise me here, please? Or maybe [~stefan.miklosovic] , [~brandon.williams] or [~jmckenzie] can advise me as you were working on removing the Windows support classes? Trying not to break users. was (Author: e.dimitrova): So the property I am wondering about is sstable_preemptive_open_interval. It seems we have a [check|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/io/sstable/SSTableRewriter.java#L109] for being less than 0, I can see that for Windows we were setting it to -1 [here|https://github.com/apache/cassandra/commit/da47849b50daa0580f2cb4264bcee8a75140eb05#diff-054af65b8d690b0fddc3e0a4ef05a80d8f1d6689b4f77912795fec019200666cL2806] before the windows support was removed, not anymore. Negatives are not documented anywhere and in 4.1 we cannot set it to negatives anymore (after CASSANDRA-15234). Not sure whether we should add a converter similar to other properties that if this one was set to negative value with the old name sstable_preemptive_open_interval_in_mb we should assign null with the new name post 4.1 and use that one. I suspect it was left undocumented and advanced option? Or was -1 only for Windows?? [~benedict] I see you added this one long time ago in CASSANDRA-6916, do you think you can advise me here, please? Or maybe [~stefan.miklosovic] , [~brandon.williams] or [~jmckenzie] can advise as you were working on removing the windows support? Trying not to break users. > Validate that JMX updates properly any properties that were moved to the new > config classes > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval, user_defined_functions_fail_timeout, > write_request_timeout, cdc_free_space_check_interval, roles_validity, > internode_streaming_tcp_user_timeout, gc_warn_threshold, > range_request_timeout, credentials_update_interval, truncate_request_timeout, > cas_contention_timeout, periodic_commitlog_sync_lag_block, > streaming_state_expires, repair_request_timeout, permissions_update_interval, > dynamic_snitch_reset_interval, internode_tcp_connect_timeout, > paxos_purge_grace_period, dynamic_snitch_update_interval,
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[
https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17571337#comment-17571337
]
Andres de la Peña edited comment on CASSANDRA-17737 at 7/26/22 10:41 AM:
-
So far it looks good to me, +1. I've seen that there were some previous typos
in the way the "threshold" word was written on {{{}GCInspector{}}}, including
user-visible error messages. We could fix that nit on commit, [this
way|https://github.com/adelapena/cassandra/commit/6704589093c6d2f869e4f2b63392432d22f2f9ab].
The plan is to include the next batches of commits into this same ticket, right?
was (Author: adelapena):
Looks good to me, +1. I've seen that there were some previous typos in the way
the "threshold" word was written on {{{}GCInspector{}}}, including user-visible
error messages. We could fix that nit on commit, [this
way|https://github.com/adelapena/cassandra/commit/6704589093c6d2f869e4f2b63392432d22f2f9ab].
> Validate that JMX updates properly any properties that were moved to the new
> config classes
> ---
>
> Key: CASSANDRA-17737
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17737
> Project: Cassandra
> Issue Type: Task
> Components: Local/Config
>Reporter: Ekaterina Dimitrova
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x
>
>
> Check that any properties moved to the new types in 4.1 (Duration, Data
> Storage, Data Rate) are always updated by JMX and there are no inconsistent
> validations that might cover bugs. Validate proper update in Settings Virtual
> Table
> I branched the configCompatibilityTest in 4.1 in order to get the lists of
> properties in this
> [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137]
> - the commit won't get into the codebase, just pasting so people know how
> the lists were generated:
> *DataRateSpec* - entire_sstable_stream_throughput_outbound,
> inter_dc_stream_throughput_outbound,
> entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput,
> stream_throughput_outbound - we have setGet* tests and those seem solid, no
> issues found.
> *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh,
> request_timeout, hints_flush_period, read_request_timeout,
> index_summary_resize_interval, streaming_keep_alive_period, max_hint_window,
> roles_update_interval, user_defined_functions_fail_timeout,
> write_request_timeout, cdc_free_space_check_interval, roles_validity,
> internode_streaming_tcp_user_timeout, gc_warn_threshold,
> range_request_timeout, credentials_update_interval, truncate_request_timeout,
> cas_contention_timeout, periodic_commitlog_sync_lag_block,
> streaming_state_expires, repair_request_timeout, permissions_update_interval,
> dynamic_snitch_reset_interval, internode_tcp_connect_timeout,
> paxos_purge_grace_period, dynamic_snitch_update_interval,
> trace_type_query_ttl, denylist_initial_load_retry, commitlog_sync_period,
> native_transport_idle_timeout, credentials_validity,
> validation_preview_purge_head_start, repair_state_expires,
> internode_tcp_user_timeout, trace_type_repair_ttl, cache_load_timeout,
> commitlog_sync_group_window, slow_query_log_timeout,
> counter_write_request_timeout, user_defined_functions_warn_timeout
> *DataStorageSpec* -
> internode_application_send_queue_reserve_endpoint_capacity, cdc_total_space,
> networking_cache_size, commitlog_total_space,
> internode_application_send_queue_capacity, key_cache_size,
> memtable_heap_space, trickle_fsync_interval, max_hints_size_per_host,
> internode_application_receive_queue_reserve_endpoint_capacity,
> native_transport_max_frame_size, coordinator_read_size_warn_threshold ,
> internode_application_receive_queue_reserve_global_capacity,
> internode_max_message_size, file_cache_size, local_read_size_fail_threshold,
> data_disk_usage_max_disk_size, memtable_offheap_space,
> coordinator_read_size_fail_threshold, counter_cache_size,
> prepared_statements_cache_size, batchlog_replay_throttle,
> row_index_read_size_fail_threshold, index_summary_capacity,
> repair_session_space, paxos_cache_size, collection_size_fail_threshold,
> internode_application_send_queue_reserve_global_capacity, column_index_size,
> native_transport_receive_queue_capacity, sstable_preemptive_open_interval,
> max_mutation_size, min_free_space_per_drive, batch_size_fail_threshold,
> hinted_handoff_throttle, row_index_read_size_warn_threshold, max_value_size,
> column_index_cache_size, compaction_large_partition_warning_threshold,
> max_hints_file_size, collection_size_warn_threshold,
>
[jira] [Comment Edited] (CASSANDRA-17737) Validate that JMX updates properly any properties that were moved to the new config classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17570574#comment-17570574 ] Ekaterina Dimitrova edited comment on CASSANDRA-17737 at 7/25/22 2:12 AM: -- First batch of issues I discovered: * CASSANDRA-17768 - streaming_keep_alive_period is not used after CASSANDRA-16927 CEP-10 Phase 1: Refactor Streaming * Fixed 2 properties in GCInspector * Fixed non-mutable prepared_statements_cache_size_mb, key_cache_size_in_mb, counter_cache_size_in_mb to be updated in Settings Virtual Table on startup * Fixed conf.index_summary_capacity_in_mb - this is ninja fix for CASSANDRA-17735 4.0 branch as it seems the 1 line fix for 4.0 didn't make it after squash to 4.0, only 4.1 and trunk. I noticed that only after the commit unfortunately :( * Fixed two GCInspector properties - further to not being updated for Settings Virtual Table the JMX methods accept long while the Config properties are int. [4.0|https://github.com/apache/cassandra/pull/1748] [CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=17737-4.0] [4.1|https://github.com/apache/cassandra/pull/1747] [CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=C17737-4.1] [trunk|https://github.com/apache/cassandra/pull/1749] [CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=C17737-trunk] [~adelapena] , do you think you can check this batch of commits, please? I will probably add a few more tomorrow but I want to make this work incremental as I find that easier (a few more tests and one undocumented 4.0 property that I am suspicious about and if I find anything else in the meantime...) was (Author: e.dimitrova): First batch of issues I discovered: * CASSANDRA-17768 - streaming_keep_alive_period is not used after CASSANDRA-16927 CEP-10 Phase 1: Refactor Streaming * Fixed 2 properties in GCInspector * Fixed non-mutable prepared_statements_cache_size_mb, key_cache_size_in_mb, counter_cache_size_in_mb * Fixed conf.index_summary_capacity_in_mb - this is ninja fix for CASSANDRA-17735 4.0 branch as it seems the 1 line fix for 4.0 didn't make it after squash to 4.0, only 4.1 and trunk. I noticed that only after the commit unfortunately :( * Fixed two GCInspector properties - further to not being updated for Settings Virtual Table the JMX methods accept long while the Config properties are int. [4.0|https://github.com/apache/cassandra/pull/1748] [CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=17737-4.0] [4.1|https://github.com/apache/cassandra/pull/1747] [CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=C17737-4.1] [trunk|https://github.com/apache/cassandra/pull/1749] [CI|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=C17737-trunk] [~adelapena] , do you think you can check this batch of commits, please? I will probably add a few more tomorrow but I want to make this work incremental as I find that easier (a few more tests and one undocumented 4.0 property that I am suspicious about and if I find anything else in the meantime...) > Validate that JMX updates properly any properties that were moved to the new > config classes > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval,
