Hello Ashish, Thanks for the explanation, it is clear now.
Regards, Gil Le 23/06/26 02:21, Ashish Vijaywargiya a écrit : > Hello Gil, > > To be precise about where the boundary sits: today, the lookup key (e.g. > jdbc-password-lookup="mysql-ofbiz") lives as a plain attribute directly in > entityengine.xml, the same place jdbc-username and jdbc-driver already > live. EntityConfig.getJdbcPassword() reads that attribute and hands the key > to the SecretProvider SPI, which is the only thing that ever calls out to > the vault, fetches the real value, caches it, and refreshes it. > > If Typesafe Config (PR #1285) lands in the future, that key could instead > live in a HOCON override file rather than the XML attribute - but the > division of labor stays the same: Typesafe Config would only ever hold > which key to use, never the secret itself. The SPI would still be the only > piece that resolves that key into an actual value via the vault. > > So in that future setup, the SPI would simply be reading the lookup key > from Typesafe Config instead of from XML - everything downstream of that > (the vault call, caching, rotation) is unchanged. > > These were my thoughts. Please let me know if you require additional > information on this. > > Thank you! > > -- > Kind Regards, > Ashish Vijaywargiya > Vice President of Operations > *HotWax Systems* > *Enterprise open source experts* > cell: +91-9893479711 > http://www.hotwaxsystems.com > > > > On Mon, Jun 22, 2026 at 10:09 PM gil.portenseigne < > [email protected]> wrote: > > > Hi Ashish, > > > > Thanks for the clarification. I don't think I was positioning the two > > approaches as competing either. I was rather thinking about different > > scales and operational models. > > > > My point was that, in more basic deployments, retrieving secrets > > directly from the application makes a lot of sense and provides valuable > > capabilities. In larger infrastructures, secret management is often > > delegated to the platform itself, which exposes credentials to > > applications through environment variables or mounted files. > > > > I am not sure I fully understood how you envision combining the two > > approaches. Could you clarify this part? From my understanding, Typesafe > > Config with environment variable and the SecretProvider SPI seem to > > overlap somewhat in terms of configuration value resolution, so I would > > be interested to better understand the interaction between both. > > > > Regarding live secret invalidation, I agree that this is a strong > > advantage in simpler architectures where restarting an application can > > be disruptive. In Kubernetes-based environments, however, rolling > > restarts can generally be performed transparently without service > > interruption, so the operational benefit may depend on the deployment > > model. > > > > Looking forward to reading more about it, and thanks again for taking > > the time to explain your design. > > > > Thanks, > > > > Gil > > Le 22/06/26 05:04, Ashish Vijaywargiya a écrit : > > > Hello Gil, > > > > > > I looked into PR #1285 (OFBIZ-12401 & related issues like OFBIZ-9498, > > > OFBIZ-10407) more closely, and I don't think the two are in conflict — > > they > > > sit at different layers: > > > > > > Here is my take on your points: > > > > > > 1. Infra-level secret retrieval vs. the Lightbend Typesafe Config > > initiative > > > > > > - The Typesafe/Lightbend Config work is about how OFBiz assembles its > > > configuration > > > — HOCON files layered as overrides on top of the existing XML config, > > with > > > native ${VAR} / ${?VAR} substitution from the process environment. If a > > > secret is exported as an env var by something external (Vault Agent > > > sidecar, External Secrets Operator, a CI/CD pipeline, etc.) before the > > JVM > > > starts, that layer can pick it up cleanly. The actual fetching and > > > lifecycle of the secret happens entirely outside OFBiz. > > > > > > - What I've built is the opposite direction: OFBiz itself talks to the > > > vault via a small SPI at the moment a value is actually needed (DB > > password > > > resolution, a SystemProperty lookup), with a cache TTL and fallback > > chain — so > > > it works even without any sidecar/operator infra in place, and it can > > pick > > > up a rotated secret without a restart. > > > > > > Rather than positioning these as competing approaches, I'd suggest a > > > concrete convergence point: a HOCON override file could set > > > jdbc-password-lookup = "jdbc-password.mysql-ofbiz" (i.e., use Typesafe > > > Config purely as the override mechanism for which lookup key to use), > > while > > > the SecretProvider SPI still does the actual vault call, caching, and > > > rotation handling underneath. That way Typesafe Config stays focused on > > > "how config is assembled," and the secret-manager plugins stay focused on > > > "how credential-shaped values in that config get resolved and refreshed" > > — > > > instead of the two solving the same problem twice. > > > > > > 2. Live secret invalidation > > > > > > That's the one capability that's hard to get from a pure > > > env-var/restart-based setup. Most vault rotation policies (HashiCorp > > > Vault's lease TTLs, AWS Secrets Manager's automatic rotation, etc.) > > expect > > > consumers to re-fetch on a schedule, not to have credentials baked into a > > > process that only refreshes on restart. With the cache-TTL-driven SPI, a > > > rotated DB password or gateway key in the vault becomes visible to a > > > running OFBiz instance on the next refresh — no coordinated restart, no > > > brief window where the app is holding a stale credential the vault has > > > already invalidated. > > > > > > Thanks again for digging in. I feel that my work on Secret Managers/Vault > > > support in Apache OFBiz is worth merging — it's backward-compatible, > > > opt-in, and the only approach here that gives live secret rotation > > without > > > a restart. > > > > > > Please let me know your thoughts whenever you can. > > > > > > =============== > > > I was referring to Kubernetes secret operators, such as > > > https://developer.hashicorp.com/vault/docs/deploy/kubernetes/vso > > > > > > In my view, in a project configuration that includes: > > > * The client for whom the implementation is being carried out > > > * The integrator who customizes and develops the solution > > > * The operations team: which manages and monitors the infrastructure > > > > > > Some of the secrets must be managed solely by the OPS team, without > > > being shared with the integrator; so I thought that secret management > > > should take place outside of OFBiz. > > > > > > But not all project follow that structure :) > > > =============== > > > > > > I am thinking about your recent email and will get back once I have > > > additional thoughts. Thank you for sharing. > > > > > > -- > > > Kind Regards, > > > Ashish Vijaywargiya > > > Vice President of Operations > > > *HotWax Systems* > > > *Enterprise open source experts* > > > http://www.hotwaxsystems.com > > > > > > > > > > > > On Fri, Jun 19, 2026 at 8:16 PM gil.portenseigne via dev < > > > [email protected]> wrote: > > > > > > > Hello Ashish, > > > > > > > > I tried to have a look about your contribution, that a quite big one > > ;). > > > > > > > > I just wanted to point that if you intend to merge this weekend it is > > > > quite short for people to have a look :). > > > > > > > > First I had some concerns about the fact that secret management, in my > > > > experience, is an infra domain subject. There are tools which goals are > > > > to retrieve and share those secrets into application environment. > > > > > > > > With the other initiative of integrating lighbend typesafe config [1], > > > > that could allow another way of configuring OFBiz app, with environment > > > > variables support for secret. > > > > > > > > That do not go against your proposal, but I guess that both can serve > > > > the same purpose. Do you have some thought about that ? > > > > > > > > I feel like the EncryptValue value feature is for simple production > > grade > > > > architecture, for example an OFBiz instance hosted in one server. > > > > Or maybe, since I am not familiar with PCI-DSS, SOC2, etc., those > > > > present some specific requirement that make config encryption > > mandatory ? > > > > > > > > The plus I see from this implementation is the secret invalidation > > > > without restarting the application, that is quite nice. > > > > > > > > I will try to go deeper soon, looking forward to reading you and > > others! > > > > > > > > Thanks, > > > > > > > > Gil > > > > > > > > [1] https://github.com/apache/ofbiz-framework/pull/1285 > > > > > > > > Le 17/06/26 11:43, Ashish Vijaywargiya a écrit : > > > > > Hello Dev Members, > > > > > > > > > > I'd like to propose merging the changes from the following branches: > > > > > > > > > > "various-secret-manager" branch - > > > > > > > > > > > https://github.com/ashishvijaywargiya/ofbiz-framework/tree/various-secret-manager > > > > > > > > > > "plugins-various-secret-manager" branch - > > > > > > > > > > > https://github.com/ashishvijaywargiya/ofbiz-plugins/tree/plugins-various-secret-manager > > > > > > > > > > into trunk. > > > > > > > > > > I shared a detailed summary of the changes in my earlier email. In > > short, > > > > > it adds support for pluggable secret managers, a webtools "Encrypt > > Value" > > > > > screen with CSV bulk upload, and a configurable secret-value marker. > > It > > > > > will be an important feature for the Apache OFBiz project for > > > > > compliance-sensitive deployments(PCI-DSS, SOC2, etc.). > > > > > > > > > > The Secret Manager functionality is disabled by default, with the > > > > existing > > > > > passwords.properties/ENC(...) behavior used as the fallback when no > > > > > provider is configured. This means existing OFBiz installations will > > > > > continue to work exactly as before, with no changes required and no > > > > > breaking changes to current functionality. > > > > > > > > > > Unless there are concerns, I'm planning to merge these changes into > > trunk > > > > > over the coming weekend. Please take a look at the branch before > > then and > > > > > share any feedback or concerns. > > > > > > > > > > Here are the PRs for your kind reference: > > > > > > > > > > https://github.com/apache/ofbiz-framework/pull/1367 > > > > > > > > > > https://github.com/apache/ofbiz-plugins/pull/291 > > > > > > > > > > Thank you in advance for your help and review. > > > > > > > > > > I will also share the User Manual/ DIY(Do It Yourself) document for > > > > Secret > > > > > Managers setup on the OFBiz Wiki sometime in the next week. > > > > > > > > > > -- > > > > > Kind Regards, > > > > > Ashish Vijaywargiya > > > > > Vice President of Operations > > > > > *HotWax Systems* > > > > > *Enterprise open source experts* > > > > > http://www.hotwaxsystems.com > > > > > > > > > > > > > > > On Tue, Jun 16, 2026 at 8:58 PM Ashish Vijaywargiya < > > > > > [email protected]> wrote: > > > > > > > > > > > Hello Dev Members, > > > > > > > > > > > > I have updated the Secret Manager/Vault support in Apache OFBiz, > > and > > > > the > > > > > > following changes have been made in my forked version. > > > > > > > > > > > > Webtools Secret Manager Screen (EncryptValue) > > > > > > ---------------------------------------------- > > > > > > > > > > > > 1. Single-entry form with dual storage targets. > > > > > > > > > > > > The EncryptValue screen supports two storage targets via a radio > > > > button: > > > > > > SystemProperty entity and passwords.properties. For the > > SystemProperty > > > > > > target, Resource ID and Property ID are required; Lookup Key is > > > > optional > > > > > > and, when provided, wires the record to a remote secret provider. > > For > > > > the > > > > > > passwords.properties target, two sub-cases are supported: > > > > > > > > > > > > Case A (Lookup Key only): writes jdbc-password.<key>=ENC(...) to > > > > > > passwords.properties, covering the entityengine.xml > > > > jdbc-password-lookup > > > > > > use case. > > > > > > > > > > > > Case B (all four fields): writes <key>=ENC(...) to > > > > passwords.properties, > > > > > > sets <propertyId>=LOOKUP(<key>) in the source .properties file on > > > > disk, and > > > > > > flushes the OFBiz property cache — no server restart needed. > > > > > > > > > > > > 2. Dynamic labels based on target selection. > > > > > > > > > > > > Field labels update dynamically when the target radio button is > > > > switched. > > > > > > Selecting SystemProperty shows "System Resource ID > > (SystemProperty)" > > > > and > > > > > > "System Property ID (SystemProperty)"; selecting > > passwords.properties > > > > shows > > > > > > "Resource ID (Property File)" and "Property ID (Property File)". > > This > > > > > > avoids confusion about which entity or file a field refers to. > > > > > > > > > > > > 3. Form field persistence on validation errors. > > > > > > > > > > > > When a submission fails validation, the radio button selection, > > > > > > systemResourceId, systemPropertyId, and lookupKey fields are > > > > pre-populated > > > > > > with the values the user entered. The secretValue field is > > > > intentionally > > > > > > cleared on error so the plain-text password is never re-rendered > > in the > > > > > > page. > > > > > > > > > > > > 4. Stale lookup key cleanup. > > > > > > > > > > > > When Case B is used and the user changes the lookupKey for a > > property > > > > that > > > > > > already had a LOOKUP(...) reference in the source .properties file, > > > > the old > > > > > > key's entry is automatically removed from passwords.properties > > before > > > > the > > > > > > new one is written, preventing orphaned encrypted entries. > > > > > > > > > > > > 5. CSV bulk upload with security pre-scan. > > > > > > > > > > > > A CSV upload path allows multiple secrets to be created in one > > > > operation > > > > > > (target, systemResourceId, systemPropertyId, lookupKey, secretValue > > > > > > columns). The entire file is validated before any writes occur — > > the > > > > file > > > > > > is rejected as a whole if any row contains unknown targets, invalid > > > > > > identifier characters (only letters, digits, dots, hyphens, and > > > > underscores > > > > > > are allowed in identifier fields), path traversal sequences (..), > > or > > > > if the > > > > > > file exceeds 512 KB or 500 rows. The whole-file content scan uses > > > > OFBiz's > > > > > > existing SecuredUpload.isValidTextContent() sandboxing mechanism to > > > > block > > > > > > null bytes and control characters at the Unicode code-point level. > > > > > > > > > > > > 6. Input validation on form and CSV upload. > > > > > > > > > > > > Server-side guards reject a LOOKUP(...) marker typed into the > > lookupKey > > > > > > field and an ENC(...) prefix typed into the secretValue field, on > > both > > > > the > > > > > > single-entry form and the CSV bulk-upload path. A matching > > client-side > > > > > > JavaScript validator highlights the offending field and shows an > > inline > > > > > > error before the form is even submitted, preventing accidental > > misuse. > > > > > > > > > > > > Marker and Resolver Changes > > > > > > ---------------------------- > > > > > > > > > > > > 7. Configurable secret marker (default: LOOKUP). > > > > > > > > > > > > The marker used to identify secret references in property values > > > > > > (previously hardcoded as SECRET) is now configurable via > > > > > > secret.value.marker in security.properties. The default has been > > > > changed to > > > > > > LOOKUP, which aligns naturally with the existing > > systemPropertyLookup > > > > field > > > > > > name in the SystemProperty entity and the jdbc-password-lookup > > > > attribute in > > > > > > entityengine.xml. Admins can change it to any uppercase word if it > > > > collides > > > > > > with an existing property value. > > > > > > > > > > > > 8. Plain key in SystemProperty.systemPropertyLookup. > > > > > > > > > > > > Previously, the field stored a wrapped value like > > LOOKUP(smtp-key). It > > > > now > > > > > > stores the plain key (smtp-key) directly, since the field name > > already > > > > > > implies lookup semantics. EntityUtilProperties now calls the new > > > > > > SecretValueResolver.resolveKey(String) method to resolve the raw > > key > > > > > > without needing a marker wrapper. Marker-based resolution > > > > (LOOKUP(key)) in > > > > > > .properties files remains unchanged. > > > > > > > > > > > > 9. Marker config moved to security.properties. > > > > > > > > > > > > secret.value.marker and secret.cache.ttl.seconds are now > > documented in > > > > > > security.properties rather than general.properties, which is the > > more > > > > > > appropriate home for secret-management configuration. > > > > > > > > > > > > 10. Fixed marker coupling in SecretManagerServices. > > > > > > > > > > > > All hardcoded "SECRET(" string literals in the writer and parser > > have > > > > been > > > > > > replaced with SecretValueResolver.MARKER_NAME (now public), > > ensuring > > > > the > > > > > > writer and resolver always stay in sync when an admin changes the > > > > > > configured marker. > > > > > > > > > > > > 11. SecretValueResolver refactored. > > > > > > > > > > > > Cache and provider logic extracted into a new public > > resolveKey(String > > > > > > key) method; resolve(String) now delegates to it after parsing the > > > > marker. > > > > > > This eliminates duplicated cache logic and gives callers a clean > > API > > > > for > > > > > > direct raw-key resolution without constructing a synthetic > > LOOKUP(...) > > > > > > wrapper. > > > > > > > > > > > > If you wish to review the code changes, please review the URL > > below: > > > > > > > > > > > > > > > > > > https://github.com/ashishvijaywargiya/ofbiz-framework/tree/various-secret-manager > > > > > > > > > > > > Thank you. > > > > > > > > > > > > PS: I noticed that my previous email had formatting issues. > > Sharing it > > > > > > again. > > > > > > > > > > > > -- > > > > > > Kind Regards, > > > > > > Ashish Vijaywargiya > > > > > > Vice President of Operations > > > > > > *HotWax Systems* > > > > > > *Enterprise open source experts* > > > > > > http://www.hotwaxsystems.com > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Jun 15, 2026 at 5:35 PM Ashish Vijaywargiya < > > > > > > [email protected]> wrote: > > > > > > > > > > > >> Dear Dev Community, > > > > > >> > > > > > >> Over the past few days, I've been working on adding optional > > Secret > > > > > >> Manager/Vault integration to Apache OFBiz, and I'd like to share > > it > > > > with > > > > > >> the Dev list for feedback before taking it further. > > > > > >> > > > > > >> The goal is simple: today, DB passwords, JWT/login keys, and > > various > > > > > >> gateway credentials mostly live as plain text in .properties > > files or > > > > in > > > > > >> the SystemProperty entity, which is a real concern for production > > and > > > > > >> compliance-sensitive deployments (PCI-DSS, SOC2, etc.). This work > > > > gives > > > > > >> adopters a way to back those values with an external secret > > manager of > > > > > >> their choice, while leaving everything else untouched. > > > > > >> > > > > > >> To make this happen, I've added six provider plugins, each > > > > implementing a > > > > > >> small SecretProvider SPI: AWS Secrets Manager, GCP(Google Cloud > > > > Platform) > > > > > >> Secret Manager, Azure Key Vault, HashiCorp Vault by IBM, > > 1Password, > > > > and > > > > > >> Bitwarden Secrets Manager. > > > > > >> > > > > > >> On the framework side, the key additions are a > > > > > >> SecretProvider/SecretProviderFactory/FallbackSecretProvider setup > > in > > > > > >> framework/base, a ConfigCryptoUtil class for local AES-256-GCM > > > > encryption > > > > > >> of values as ENC(...) keyed by OFBIZ_MASTER_KEY, and a > > > > SecretValueResolver > > > > > >> that resolves any SECRET(key) reference in a property file or > > > > > >> SystemProperty row through the configured provider, with a > > > > configurable > > > > > >> cache TTL. > > > > > >> > > > > > >> entityengine.xml now supports jdbc-password-lookup on inline-jdbc > > > > > >> elements, which is resolved via SecretProviderFactory at startup, > > so > > > > the DB > > > > > >> password never needs to appear in entityengine.xml at all. > > Previously, > > > > > >> passwords.properties only supported plain-text values; now those > > same > > > > > >> entries can be encrypted at rest using a new generateDBPassword > > gradle > > > > > >> task, which prompts for the value via masked stdin and writes out > > the > > > > > >> ENC(...) form, so no secret ever needs to be typed on the command > > > > line or > > > > > >> land in shell history. > > > > > >> > > > > > >> SystemProperty also has a new systemPropertyLookup field so > > individual > > > > > >> properties can opt into secret-manager resolution. > > > > > >> > > > > > >> I've also added unit tests (e.g. SecretValueResolverTest) covering > > > > plain > > > > > >> values, ENC(...) values, cached lookups, and provider > > failure/fallback > > > > > >> behavior, plus a generateDBPassword gradle task with the same > > > > masked-prompt > > > > > >> approach for setting up the local DB password. > > > > > >> > > > > > >> On the impact to existing implementations/configurations, this is > > > > fully > > > > > >> backward-compatible and opt-in only: > > > > > >> > > > > > >> 1) passwords.properties / entityengine.xml – Existing plain-text > > > > > >> jdbc-password values continue to work exactly as before. Sites can > > > > > >> optionally migrate to ENC(...) (AES-256-GCM, keyed by > > > > OFBIZ_MASTER_KEY, > > > > > >> generated via generateEncryptedSecret) or to a > > jdbc-password-lookup > > > > key > > > > > >> resolved via a secret manager, but if they do nothing, behavior is > > > > > >> unchanged. > > > > > >> > > > > > >> 2) SystemProperty – The new systemPropertyLookup field is > > additive. > > > > > >> Existing rows with only systemPropertyValue (plain or ENC(...)) > > > > behave as > > > > > >> they do today. A site only sees secret-manager resolution if it > > > > explicitly > > > > > >> sets systemPropertyLookup on a row. > > > > > >> > > > > > >> 3) Secret managers – All six provider plugins (AWS, Azure, > > HashiCorp, > > > > > >> GCP, 1Password, Bitwarden) are optional. None are enabled by > > default, > > > > with > > > > > >> no new mandatory dependencies in core. You can enable one > > > > plugin/provider > > > > > >> at a time by changing values in the respective plugin's > > > > ofbiz-component.xml > > > > > >> file. > > > > > >> > > > > > >> 4) Fallback chain – Lookup → remote secret manager (if > > configured) → > > > > > >> local ENC(...)/plain value. If the remote provider is unreachable > > or > > > > > >> unconfigured, OFBiz falls back to the existing file/entity-based > > > > value, > > > > > >> resulting in no new failure mode for sites that do not adopt this > > > > > >> functionality. > > > > > >> > > > > > >> 5) Testing – The new resolver, crypto utility, and provider > > fallback > > > > > >> logic are all covered by unit tests, so existing config-loading > > paths > > > > > >> remain verified alongside the new optional ones. > > > > > >> > > > > > >> Here is the source code for your kind review: > > > > > >> > > > > > >> Framework related changes: > > > > > >> > > > > > >> > > > > > > https://github.com/ashishvijaywargiya/ofbiz-framework/tree/various-secret-manager > > > > > >> > > > > > >> Plugins related changes: > > > > > >> > > > > > >> > > > > > > https://github.com/ashishvijaywargiya/ofbiz-plugins/tree/plugins-various-secret-manager > > > > > >> > > > > > >> Further work: > > > > > >> > > > > > >> 1) I am working on adding an Admin Screen in webtools where a > > user can > > > > > >> create secrets either in the SystemProperty entity or the > > > > password.properties > > > > > >> file. I will provide an option where a user can create multiple > > > > secrets > > > > > >> in a CSV file and upload them in bulk. The AES algorithm will > > encrypt > > > > > >> the password and put it either in SystemProperty or > > > > password.properties > > > > > >> file. > > > > > >> > > > > > >> 2) Another item on the roadmap is secret key/password rotation > > > > handling > > > > > >> from the remote service, with the corresponding values updated in > > > > > >> passwords.properties or the relevant SystemProperty records. Many > > > > > >> third-party services expect support for secret key rotation at > > regular > > > > > >> intervals for security reasons. For now, we won't provide support > > for > > > > > >> database password rotation in the Secret Manager, since that would > > > > also > > > > > >> require handling of the user password rotation on the database > > side > > > > itself > > > > > >> (MySQL, PostgreSQL, etc.). > > > > > >> > > > > > >> 3) Merge the generateDBPassword and generateEncryptedSecret Gradle > > > > tasks > > > > > >> and make them one, i.e generateEncryptedSecret. > > > > > >> > > > > > >> Please let me know your thoughts/feedback on this work. I will > > > > quickly accommodate > > > > > >> your thoughts/ideas in the current implementation. > > > > > >> > > > > > >> I'm planning to commit the current work to the Apache OFBiz > > > > > >> project(trunk) in the next few days. > > > > > >> > > > > > >> Thank you. > > > > > >> > > > > > >> -- > > > > > >> Kind Regards, > > > > > >> Ashish Vijaywargiya > > > > > >> Vice President of Operations > > > > > >> *HotWax Systems* > > > > > >> *Enterprise open source experts* > > > > > >> http://www.hotwaxsystems.com > > > > > >> > > > > > >> > > > > > >
