[
https://issues.apache.org/jira/browse/CASSANDRA-20749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18004793#comment-18004793
]
Paulo Motta commented on CASSANDRA-20749:
-----------------------------------------
Pushed a patch to this [PR|https://github.com/apache/cassandra/pull/4231]. This
uses similar logic as the one introduced by CASSANDRA-17166 to override
arbitrary configurations via system properties and logs warnings when settings
are overridden via environment variables. Jenkins CI is
[here|https://ci-cassandra.apache.org/view/patches/job/Cassandra-devbranch-5/320/]
- looks relatively good but the build was aborted, not sure the state of
jenkins for trunk currently CI.
This feature is disabled by default via system property
{{-Dcassandra.config.allow_system_properties=false}} - I documented both this
and {{-Dcassandra.config.allow_environment_variables=false}} on
{{{}conf/jvm-server.options{}}}.
This patch also fix the limitation from CASSANDRA-17166 that did not support
overriding collection config types via system properties (ie. set/map),
documented by [this
comment|https://github.com/apache/cassandra/blob/cassandra-5.0/test/unit/org/apache/cassandra/config/YamlConfigurationLoaderTest.java#L136].
After this patch it's possible to specify collection types in system
properties as JSON strings.
*Usage Description*
* Simple configurations that take scalar types (ie. non-collection) like
number or text can be overridden via {{{}CASS_<simple_config_name>{}}}, for
example, {{uuid_sstable_identifiers_enabled}} can be overridden via the
environment variable {{{}CASS_UUID_SSTABLE_IDENTIFIERS_ENABLED{}}}.
* Nested configurations that take scalar values, for example,
{{replica_filtering_protection.cached_rows_warn_threshold}} can be overridden
by using {{__ }}(double underscores) as nested property separator. So this
property could be overridden via
{{{}CASS_REPLICA_FILTERING_PROTECTION{}}}{_}{{{}_CACHED_ROWS_WARN_THRESHOLD{}}}
environment variable.
* Configurations that take collection types (sets, maps) can only be
overridden as a "whole" (ie. no partial updates), and should be specified as a
json string.
** For example, the complex set property {{table_properties_warned}} could be
overridden via the environment variable
{{{}CASS_TABLE_PROPERTIES_WARNED="[\"sstable_preemptive_open_interval_in_mb\",
\"index_summary_resize_interval_in_minutes\"]"{}}}.
** For example, the collection map property {{startup_checks}} could be
overridden with {{{}CASS_STARTUP_CHECKS="{\"check_data_resurrection\":
{\"enabled\": \"true\", \"heartbeat_file\":
\"/var/lib/cassandra/data/cassandra-heartbeat\"{}}}"}}.
*Execution Example*
Running with this enabled on a docker image will print the following output
{noformat}
docker run -e CASS_ALTER_TABLE_ENABLED=false -e
CASS_MAXIMUM_TIMESTAMP_WARN_THRESHOLD=12h -e CASS_ZERO_TTL_ON_TWCS_WARNED=false
-e CASS_DEFAULT_SECONDARY_INDEX_ENABLED=false cassandra-test:latest
WARN [main] 2025-07-11 21:53:40,026 YamlConfigurationLoader.java:194 - Detected
environment variable CASS_DEFAULT_SECONDARY_INDEX_ENABLED=false override for
cassandra configuration 'default_secondary_index_enabled' (ignored if setting
does not exist).
WARN [main] 2025-07-11 21:53:40,183 YamlConfigurationLoader.java:194 - Detected
environment variable CASS_ZERO_TTL_ON_TWCS_WARNED=false override for cassandra
configuration 'zero_ttl_on_twcs_warned' (ignored if setting does not exist).
WARN [main] 2025-07-11 21:53:40,183 YamlConfigurationLoader.java:194 - Detected
environment variable CASS_ALTER_TABLE_ENABLED=false override for cassandra
configuration 'alter_table_enabled' (ignored if setting does not exist).
WARN [main] 2025-07-11 21:53:40,183 YamlConfigurationLoader.java:194 - Detected
environment variable CASS_MAXIMUM_TIMESTAMP_WARN_THRESHOLD=12h override for
cassandra configuration 'maximum_timestamp_warn_threshold' (ignored if setting
does not exist).
INFO [main] 2025-07-11 21:53:40,227 Config.java:1352 - Node
configuration:[allocate_tokens_for_keyspace=null;
allocate_tokens_for_local_replication_factor=3;
allow_extra_insecure_udfs=false; allow_filtering_enabled=true;
allow_insecure_udfs=false; alter_table_enabled=false;
audit_logging_options=AuditLogOptions{enabled=false, logger='BinAuditLogger',
included_keyspaces='',
excluded_keyspaces='system,system_schema,system_virtual_schema',
included_categories='', excluded_categories='', included_users='',
excluded_users='', audit_logs_dir='/opt/cassandra/logs/audit',
archive_command='', roll_cycle='HOURLY', block=true,
max_queue_weight=268435456, max_log_size=17179869184, max_archive_retries=10};
auth_cache_warming_enabled=false; auth_read_consistency_level=LOCAL_QUORUM;
auth_write_consistency_level=EACH_QUORUM;
authenticator=AllowAllAuthenticator{}; authorizer=AllowAllAuthorizer{};
auto_bootstrap=true; auto_hints_cleanup_enabled=false;
auto_optimise_full_repair_streams=false;
auto_optimise_inc_repair_streams=false;
auto_optimise_preview_repair_streams=false; auto_snapshot=true;
auto_snapshot_ttl=null; autocompaction_on_startup_enabled=true;
automatic_sstable_upgrade=false; available_processors=-1;
back_pressure_enabled=false; back_pressure_strategy=null;
batch_size_fail_threshold=50KiB; batch_size_warn_threshold=5KiB;
batchlog_endpoint_strategy=random_remote; batchlog_replay_throttle=1024KiB;
block_for_peers_in_remote_dcs=false; block_for_peers_timeout_in_secs=10;
broadcast_address=192.168.150.121; broadcast_rpc_address=192.168.150.121;
buffer_pool_use_heap_if_exhausted=false; cache_load_timeout=30s;
cas_contention_timeout=1000ms; cdc_block_writes=true; cdc_enabled=false;
cdc_free_space_check_interval=250ms; cdc_on_repair_enabled=true;
cdc_raw_directory=null; cdc_total_space=0MiB;
check_for_duplicate_rows_during_compaction=true;
check_for_duplicate_rows_during_reads=true;
cidr_authorizer=AllowAllCIDRAuthorizer; client_encryption_options=<REDACTED>;
client_error_reporting_exclusions=SubnetGroups{subnets=[]};
client_request_size_metrics_enabled=true; cluster_name=Test Cluster;
collection_size_fail_threshold=null; collection_size_warn_threshold=null;
column_index_cache_size=2KiB; column_index_size=null;
column_value_size_fail_threshold=null; column_value_size_warn_threshold=null;
columns_per_table_fail_threshold=-1; columns_per_table_warn_threshold=-1;
commit_failure_policy=stop; commitlog_compression=null;
commitlog_directory=null; commitlog_disk_access_mode=legacy;
commitlog_max_compression_buffers_in_pool=3; commitlog_segment_size=32MiB;
commitlog_sync=periodic; commitlog_sync_group_window=0ms;
commitlog_sync_period=10000ms; commitlog_total_space=null;
compact_tables_enabled=true; compaction_throughput=64MiB/s;
concurrent_compactors=null; concurrent_counter_writes=32;
concurrent_index_builders=2; concurrent_materialized_view_builders=1;
concurrent_materialized_view_writes=32; concurrent_merkle_tree_requests=0;
concurrent_reads=32; concurrent_validations=0; concurrent_writes=32;
consecutive_message_errors_threshold=1;
coordinator_read_size_fail_threshold=null;
coordinator_read_size_warn_threshold=null;
corrupted_tombstone_strategy=disabled; counter_cache_keys_to_save=2147483647;
counter_cache_save_period=7200s; counter_cache_size=null;
counter_write_request_timeout=5000ms; cql_start_time=REQUEST;
credentials_cache_active_update=false; credentials_cache_max_entries=1000;
credentials_update_interval=null; credentials_validity=2000ms;
crypto_provider=org.apache.cassandra.security.DefaultCryptoProvider{fail_on_missing_provider=false};
data_disk_usage_max_disk_size=null;
data_disk_usage_percentage_fail_threshold=-1;
data_disk_usage_percentage_warn_threshold=-1;
data_file_directories=[Ljava.lang.String;@4362d7df; default_compaction=null;
default_keyspace_rf=1; default_secondary_index=legacy_local_table;
default_secondary_index_enabled=false; denylist_consistency_level=QUORUM;
denylist_initial_load_retry=5s; denylist_max_keys_per_table=1000;
denylist_max_keys_total=10000; denylist_range_reads_enabled=true;
denylist_reads_enabled=true; denylist_refresh=600s;
denylist_writes_enabled=true; diagnostic_events_enabled=false;
disk_access_mode=mmap_index_only; disk_failure_policy=stop;
disk_optimization_estimate_percentile=0.95;
disk_optimization_page_cross_chance=0.1; disk_optimization_strategy=ssd;
drop_compact_storage_enabled=false; drop_keyspace_enabled=true;
drop_truncate_table_enabled=true; dump_heap_on_uncaught_exception=false;
dynamic_data_masking_enabled=false; dynamic_snitch=true;
dynamic_snitch_badness_threshold=1.0; dynamic_snitch_reset_interval=600000ms;
dynamic_snitch_update_interval=100ms; endpoint_snitch=SimpleSnitch;
enforce_native_deadline_for_hints=false;
entire_sstable_inter_dc_stream_throughput_outbound=24MiB/s;
entire_sstable_stream_throughput_outbound=24MiB/s;
failure_detector=FailureDetector; fields_per_udt_fail_threshold=-1;
fields_per_udt_warn_threshold=-1; file_cache_enabled=false;
file_cache_round_up=null; file_cache_size=null; flush_compression=fast;
force_new_prepared_statement_behaviour=false;
full_query_logging_options=FullQueryLoggerOptions{log_dir='',
archive_command='', roll_cycle='HOURLY', block=true,
max_queue_weight=268435456, max_log_size=17179869184}; gc_log_threshold=200ms;
gc_warn_threshold=1s; group_by_enabled=true; heap_dump_path=heapdump;
hint_window_persistent_enabled=true; hinted_handoff_disabled_datacenters=[];
hinted_handoff_enabled=true; hinted_handoff_throttle=1024KiB;
hints_compression=null; hints_directory=null; hints_flush_period=10000ms;
ideal_consistency_level=null; in_select_cartesian_product_fail_threshold=-1;
in_select_cartesian_product_warn_threshold=-1; incremental_backups=false;
index_summary_capacity=null; index_summary_resize_interval=60m;
initial_range_tombstone_list_allocation_size=1; initial_token=null;
inter_dc_stream_throughput_outbound=24MiB/s; inter_dc_tcp_nodelay=false;
internode_application_receive_queue_capacity=4MiB;
internode_application_receive_queue_reserve_endpoint_capacity=128MiB;
internode_application_receive_queue_reserve_global_capacity=512MiB;
internode_application_send_queue_capacity=4MiB;
internode_application_send_queue_reserve_endpoint_capacity=128MiB;
internode_application_send_queue_reserve_global_capacity=512MiB;
internode_authenticator=null; internode_compression=dc;
internode_error_reporting_exclusions=SubnetGroups{subnets=[]};
internode_max_message_size=null; internode_socket_receive_buffer_size=0B;
internode_socket_send_buffer_size=0B;
internode_streaming_tcp_user_timeout=300s; internode_tcp_connect_timeout=2s;
internode_tcp_user_timeout=30s; internode_timeout=true;
intersect_filtering_query_enabled=true; intersect_filtering_query_warned=true;
items_per_collection_fail_threshold=-1; items_per_collection_warn_threshold=-1;
key_cache_invalidate_after_sstable_deletion=false;
key_cache_keys_to_save=2147483647; key_cache_migrate_during_compaction=true;
key_cache_save_period=4h; key_cache_size=null; keyspaces_fail_threshold=-1;
keyspaces_warn_threshold=-1; listen_address=192.168.150.121;
listen_interface=null; listen_interface_prefer_ipv6=false;
listen_on_broadcast_address=false; local_read_size_fail_threshold=null;
local_read_size_warn_threshold=null; local_system_data_file_directory=null;
log_out_of_token_range_requests=true; materialized_views_enabled=false;
materialized_views_per_table_fail_threshold=-1;
materialized_views_per_table_warn_threshold=-1;
max_concurrent_automatic_sstable_upgrades=1; max_hint_window=3h;
max_hints_delivery_threads=2; max_hints_file_size=128MiB;
max_hints_size_per_host=0B; max_mutation_size=null;
max_space_usable_for_compactions_in_percentage=0.95; max_streaming_retries=3;
max_top_size_partition_count=10; max_top_tombstone_partition_count=10;
max_value_size=256MiB; maximum_replication_factor_fail_threshold=-1;
maximum_replication_factor_warn_threshold=-1;
maximum_timestamp_fail_threshold=null; maximum_timestamp_warn_threshold=12h;
memtable=org.apache.cassandra.config.Config$MemtableOptions@615f972;
memtable_allocation_type=heap_buffers; memtable_cleanup_threshold=null;
memtable_flush_writers=0; memtable_heap_space=null;
memtable_offheap_space=null; min_free_space_per_drive=50MiB;
min_tracked_partition_size=1MiB; min_tracked_partition_tombstone_count=5000;
minimum_replication_factor_fail_threshold=-1;
minimum_replication_factor_warn_threshold=-1;
minimum_timestamp_fail_threshold=null; minimum_timestamp_warn_threshold=null;
native_transport_allow_older_protocols=true;
native_transport_flush_in_batches_legacy=false;
native_transport_idle_timeout=0ms; native_transport_max_auth_threads=4;
native_transport_max_backoff_on_queue_overload=200ms;
native_transport_max_concurrent_connections=-1;
native_transport_max_concurrent_connections_per_ip=-1;
native_transport_max_frame_size=16MiB; native_transport_max_message_size=null;
native_transport_max_request_data_in_flight=null;
native_transport_max_request_data_in_flight_per_ip=null;
native_transport_max_requests_per_second=1000000;
native_transport_max_threads=128;
native_transport_min_backoff_on_queue_overload=10ms;
native_transport_port=9042; native_transport_port_ssl=null;
native_transport_queue_max_item_age_threshold=1.7976931348623157E308;
native_transport_rate_limiting_enabled=false;
native_transport_receive_queue_capacity=1MiB;
native_transport_throw_on_overload=false; native_transport_timeout=12s;
network_authorizer=AllowAllNetworkAuthorizer{}; networking_cache_size=null;
non_partition_restricted_index_query_enabled=true; num_tokens=16;
otc_backlog_expiration_interval_ms=200;
otc_coalescing_enough_coalesced_messages=8; otc_coalescing_strategy=DISABLED;
otc_coalescing_window_us=200; page_size_fail_threshold=-1;
page_size_warn_threshold=-1; partition_denylist_enabled=false;
partition_keys_in_select_fail_threshold=-1;
partition_keys_in_select_warn_threshold=-1; partition_size_fail_threshold=null;
partition_size_warn_threshold=null; partition_tombstones_fail_threshold=-1;
partition_tombstones_warn_threshold=-1;
partitioner=org.apache.cassandra.dht.Murmur3Partitioner; paxos_cache_size=null;
paxos_contention_max_wait=null; paxos_contention_min_delta=null;
paxos_contention_min_wait=null; paxos_contention_wait_randomizer=null;
paxos_on_linearizability_violations=ignore; paxos_purge_grace_period=60s;
paxos_repair_enabled=true; paxos_repair_parallelism=-1;
paxos_state_purging=null; paxos_topology_repair_no_dc_checks=false;
paxos_topology_repair_strict_each_quorum=false; paxos_variant=v1;
periodic_commitlog_sync_lag_block=null; permissions_cache_active_update=false;
permissions_cache_max_entries=1000; permissions_update_interval=null;
permissions_validity=2000ms; phi_convict_threshold=8.0;
prepared_statements_cache_size=null; range_request_timeout=10000ms;
range_tombstone_list_growth_factor=1.5;
read_before_write_list_operations_enabled=true;
read_consistency_levels_disallowed=[]; read_consistency_levels_warned=[];
read_request_timeout=5000ms; read_thresholds_enabled=false;
reject_out_of_token_range_requests=false;
reject_repair_compaction_threshold=2147483647;
repair_command_pool_full_strategy=queue; repair_command_pool_size=0;
repair_request_timeout=120000ms; repair_session_max_tree_depth=null;
repair_session_space=null; repair_state_expires=3d; repair_state_size=100000;
repaired_data_tracking_for_partition_reads_enabled=false;
repaired_data_tracking_for_range_reads_enabled=false;
report_unconfirmed_repaired_data_mismatches=false; request_timeout=10000ms;
role_manager=CassandraRoleManager{}; roles_cache_active_update=false;
roles_cache_max_entries=1000; roles_update_interval=null;
roles_validity=2000ms;
row_cache_class_name=org.apache.cassandra.cache.OHCProvider;
row_cache_keys_to_save=2147483647; row_cache_save_period=0s;
row_cache_size=0MiB; row_index_read_size_fail_threshold=null;
row_index_read_size_warn_threshold=null; rpc_address=0.0.0.0;
rpc_interface=null; rpc_interface_prefer_ipv6=false; rpc_keepalive=true;
sai_frozen_term_size_fail_threshold=8KiB;
sai_frozen_term_size_warn_threshold=1KiB;
sai_options=org.apache.cassandra.config.StorageAttachedIndexOptions@6058e535;
sai_sstable_indexes_per_query_fail_threshold=-1;
sai_sstable_indexes_per_query_warn_threshold=32;
sai_string_term_size_fail_threshold=8KiB;
sai_string_term_size_warn_threshold=1KiB;
sai_vector_term_size_fail_threshold=32KiB;
sai_vector_term_size_warn_threshold=16KiB; sasi_indexes_enabled=false;
saved_caches_directory=null; scripted_user_defined_functions_enabled=false;
secondary_indexes_enabled=true; secondary_indexes_per_table_fail_threshold=-1;
secondary_indexes_per_table_warn_threshold=-1;
seed_provider=org.apache.cassandra.locator.SimpleSeedProvider{seeds=192.168.150.121};
server_encryption_options=<REDACTED>; severity_during_decommission=0.0;
simplestrategy_enabled=true; skip_paxos_repair_on_topology_change=false;
skip_paxos_repair_on_topology_change_keyspaces=[];
skip_stream_disk_space_check=false; slow_query_log_timeout=500ms;
snapshot_before_compaction=false; snapshot_links_per_second=0;
snapshot_on_duplicate_row_detection=false;
snapshot_on_repaired_data_mismatch=false; ssl_storage_port=7001;
sstable_preemptive_open_interval=50MiB;
sstable_read_rate_persistence_enabled=false; start_native_transport=true;
startup_checks={}; storage_compatibility_mode=CASSANDRA_4; storage_port=7000;
stream_entire_sstables=true; stream_throughput_outbound=24MiB/s;
stream_transfer_task_timeout=12h; streaming_connections_per_host=1;
streaming_keep_alive_period=300s; streaming_slow_events_log_timeout=10s;
streaming_state_expires=3d; streaming_state_size=40MiB;
streaming_stats_enabled=true; table_properties_disallowed=[];
table_properties_ignored=[]; table_properties_warned=[];
tables_fail_threshold=-1; tables_warn_threshold=-1;
tombstone_failure_threshold=100000; tombstone_warn_threshold=1000;
top_partitions_enabled=true; trace_type_query_ttl=1d; trace_type_repair_ttl=7d;
transfer_hints_on_decommission=true; transient_replication_enabled=false;
transparent_data_encryption_options=org.apache.cassandra.config.TransparentDataEncryptionOptions@66238be2;
traverse_auth_from_root=false; trickle_fsync=false;
trickle_fsync_interval=10240KiB; truncate_request_timeout=60000ms;
uncompressed_tables_enabled=true;
unlogged_batch_across_partitions_warn_threshold=10;
use_creation_time_for_hint_ttl=true; use_deterministic_table_id=false;
use_offheap_merkle_trees=true; use_statements_enabled=true;
user_defined_functions_enabled=false;
user_defined_functions_fail_timeout=1500ms;
user_defined_functions_threads_enabled=true;
user_defined_functions_warn_timeout=500ms; user_function_timeout_policy=die;
user_timestamps_enabled=true; uuid_sstable_identifiers_enabled=false;
validation_preview_purge_head_start=3600s; vector_dimensions_fail_threshold=-1;
vector_dimensions_warn_threshold=-1; vector_type_enabled=true;
windows_timer_interval=0; write_consistency_levels_disallowed=[];
write_consistency_levels_warned=[]; write_request_timeout=2000ms;
zero_ttl_on_twcs_enabled=true; zero_ttl_on_twcs_warned=false]
{noformat}
> Allow overriding arbitrary settings via environment variables
> -------------------------------------------------------------
>
> Key: CASSANDRA-20749
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20749
> Project: Apache Cassandra
> Issue Type: Improvement
> Components: Local/Config
> Reporter: Paulo Motta
> Assignee: Paulo Motta
> Priority: Normal
>
> Allow arbitrary cassandra.yaml settings to be overriden via environment
> variables. This allows performing runtime overrides to a cassandra node
> configuration without updating cassandra.yaml, what can be useful to test
> isolated changes or in docker environments where runtime environment
> variables can be supplied via the {{--env}} flag.
> This should transparently override arbitrary configuration supported in
> cassandra.yaml via environment variables loaded during startup matching the
> format {{{}CASS_{CASSANDRA_YAML_CONFIG_NAME{}}}}. So for example, to override
> the {{cassandra.yaml}} configuration cdc_enabled, this can be overriden via
> the environment variable {{{}CASS_CDC_ENABLED{}}}, the cassandra.yaml setting
> {{key_cache_save_period}} can be overriden via {{CASS_KEY_CACHE_SAVE_PERIOD}}
> and so on. This should support both simple and complex properties.
> This is analogous to the functionality added by CASSANDRA-17166 to override
> arbitrary configurations via system properties enabled by the flag
> {{{}-Dcassandra.config.allow_system_properties{}}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
