Hi Robert, I'm revisiting this old conversation because I believe this topic is still important and deserves our attention.
I agree with all your points, particularly the tight coupling between a required Polaris entity and a secret. I would also like to add two limitations: 1) Only one secret manager can be active at a time. This constraint prevents using multiple secret references simultaneously (e.g., I cannot read both a Vault secret reference and an AWS Secrets Manager reference in the same Polaris instance). I think we need to switch to a system where a single manager is capable of fetching secrets from multiple sources. 2) Existing system-level secrets, defined via configuration, were left out. Specifically, AWS and GCP secrets, as well as the JWT broker private key and shared secret, are currently not handled by the secrets manager, and thus, can only be rotated by changing the configuration and restarting the server. It would be better if those secrets could be stored in a source managed by the secrets manager. Lack of a production-ready implementation hinders the feature and prevents its adoption. Therefore, I believe Polaris should ship with out-of-the-box support for multiple sources, e.g.: 1) Polaris configuration, e.g. "urn:polaris-secret:polaris-config:my-secret-key" 2) Metastore, e.g. "urn:polaris-secret:metastore:realm1:my-secret-key" 3) Kubernetes secrets, e.g. "urn:polaris-secret:k8s:namespace1:my-secret-name" 4) Same for Vault, AWS Secrets Manager, etc. Orthogonal to the secret source is the secret format. As you mentioned, some secrets are actually pairs (e.g. key pairs, a token + its expiration time, etc.); they can also come in different shapes (plain text, PEM-encoded, binary, files on disk, etc.). Each source should be able to support each of these layouts transparently. Thanks, Alex On Thu, Jul 17, 2025 at 11:12 AM Robert Stupp <[email protected]> wrote: > > Hi all, > > We now have an interface `UserSecretsManager` for safe storage of > secrets (thanks Dennis!). > > With that interface we just have one implementation named > `UnsafeInMemorySecretsManager`, which is pretty much what the name > says. For posterity: it does not persist secrets but only keeps those > in memory. This means, all secrets a user thinks are stored, are > effectively lost when the single Polaris instance is stopped. Or very > clear: `UnsafeInMemorySecretsManager` is NOT for production use. > > Even clearer: we have no `UserSecretsManager` implementation that can > be used by our users, meaning that our users cannot use for example > "IMPLICIT authentication". > > I think this topic deserves some more attention and volunteers for > implementations against actual secret managers (hyperscalers and > Vault). > > Honestly, I mostly missed the addition of the `UserSecretsManager` > interface and I think we can improve it. > > The interface requires an entity (it's ID). I don't think that this > tight relationship is necessary in the `UserSecretsManager` interface. > I think we should also allow users to directly provide a secret, which > I don't see being implemented (or even possible?) today. > Regular rotation of secrets is a requirement. A secret for S3 for > example consists of the "access key ID" and the "secret key" - the > actual secret is therefore a tuple of both values. For that it's > necessary to have a composite object stored and retrieved atomically > in secret managers. > > Given that there is no production implementation, it is fine to change it. > > Thoughts on these topics? > > Robert
