> My suggestion / question was that we might / could abstract away the configuration store. A store might also be remote. Once done, we might have a custom ConfigurationLoader on Cassandra's classpath which will fetch the configuration remotely - from the very store we would have remotely too. That way, nothing would ever be on disk.
Hi Stefan, I reflected about our discussion and I think the local configuration store should not be pluggable. The configuration store is intentionally local and not pluggable because it serves a different purpose than the ConfigurationProvider. The store is a local materialization cache used by the LifecycleProvider to start Cassandra even when the remote provider is unavailable. When a custom Cassandra ConfigurationLoader is used the materialized artifacts from disk may not be needed to start Cassandra, but the local store is still needed to cache the overlay config and its hash metadata. The cached overlay hash is required when sending updates to the remote provider, even when the provider is reachable. Without the local copy maintained in the configuration store there would be no overlay hash available to perform PATCH operations against the provider. For storing configuration remotely, the path is to implement a custom ConfigurationProvider that reads and writes to the remote system. Cheers, Paulo On Tue, 17 Mar 2026 at 16:43 Štefan Miklošovič <[email protected]> wrote: > To add my 2c, just bringing it to the public what I was talking with > Paulo privately. > > We also have a pluggable way to fetch Config in Cassandra, > YamlConfigurationLoader is just the default, it implements > ConfigurationLoader interface. So there is already a way to fetch > configs remotely. > > The CEP, as it is now, looks at LocalConfigurationStore (looking into > the chart in CEP) as at something which is truly _local_, interacts > with a disk. What is currently pluggable is ConfigurationProvider, so > you might have your configs in an external system. > > But the _store_ is not pluggable, it is local only. > > My suggestion / question was that we might / could abstract away the > configuration store. A store might also be remote. Once done, we might > have a custom ConfigurationLoader on Cassandra's classpath which will > fetch the configuration remotely - from the very store we would have > remotely too. That way, nothing would ever be on disk. > > I leave this up to Paulo how / if he addresses that. While I can > definitely see it might be handy to abstract away the whole > configuration store, it is not something which would be in particular > a blocker. I just sense that once we ship this without the ability of > having a store remotely, the very next ticket will be "but I want to > have my store remotely". > > On Tue, Mar 17, 2026 at 5:32 PM Paulo Motta <[email protected]> wrote: > > > > Hi everyone, > > > > I'd like to propose CEP-62: Cassandra Configuration Management via > Sidecar for discussion by the community. > > > > CASSSIDECAR-266[1] introduced Cassandra process lifecycle management > capabilities to Sidecar, giving operators the ability to start and stop > Cassandra instances programmatically. However, Sidecar currently has no way > to manipulate the configuration files that those instances consume at > startup. > > > > Many Cassandra settings (memtable configuration, SSTable settings, > storage_compatibility_mode) cannot be modified at runtime via JMX/CQL and > must be set in cassandra.yaml or JVM options files, requiring a restart to > take effect. Managing these files manually or through custom tooling is > cumbersome and lacks a stable API. > > > > This CEP extends Sidecar's lifecycle management by adding configuration > management capabilities for persisted configuration artifacts. It > introduces a REST API for reading and updating cassandra.yaml and JVM > options, a pluggable ConfigurationProvider abstraction for integration with > centralized configuration systems (etcd, Consul, or custom backends), and > version-aware validation to prevent startup failures. > > > > This CEP also serves as a prerequisite for future Cassandra upgrades via > Sidecar. For example, upgrading from Cassandra 4 to Cassandra 5 requires > updating storage_compatibility_mode in cassandra.yaml. The configuration > management capabilities introduced here will enable Sidecar to orchestrate > such upgrades by updating configuration artifacts alongside binary version > changes. > > > > The CEP is linked here: > https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-62%3A+Cassandra+Configuration+Management+via+Sidecar > > > > Looking forward to your feedback! > > > > Thanks, > > > > Paulo > > > > [1] - https://issues.apache.org/jira/browse/CASSSIDECAR-266 >
