Being based primarily in Docker containers and having experience with both Kubernetes (where secrets such as KESTORE_PASSWORD can be injected as environment variables or files) and Docker Swarm (which only handles secrets as environment variables), I'd have definitely been wanting this before moving from Variables to Parameters if I was still in Swarm (or Docker Compose/straight up Docker).
It's certainly possible to script creating/updating Parameters via the Toolkit/NiPyAPI, but in Docker Swarm that's not so easy (whereas it's possible as a Job in Kubernetes, for example). So environment variables could save the day in that instance. I guess one likely problem (but no different to how I guess the Variable Registry uses env vars) would be how NiFi will handle changes to the env vars - does it: - ignore them until instance restart, which could lead to maintainer confusion (I've changed KEYSTORE_PASSWORD in the env but things are still failing in NiFi) - alert the maintainer to the fact that the env var has changed and a Parameter needs updating, with the new value being used after all associated processors/controllers have been restarted - automatically attempt to update the parameters by restarting all associated processors/controllers, which I'd guess would be a bit dangerous for interrupting in-flow data, etc. --- *Chris Sampson* IT Consultant [email protected] <https://www.naimuri.com/> On Mon, 19 Oct 2020 at 19:35, Bryan Bende <[email protected]> wrote: > Access to environment variables directly from expression language is > not being removed. > > The discussion is about whether a parameter value should be able to > use expression language to reference an environment variable. > > For example, processor property has #{keystore.password} -> parameter > "keystore.password" has value "${KEYSTORE_PASSWORD}" which then gets > the password from an environment variable. > > > > On Mon, Oct 19, 2020 at 2:14 PM [email protected] <[email protected]> > wrote: > > > > Chad, > > > > So far I thought that only the NiFi variables are deprecated and access > to environment variables will still be possible. > > > > If this is not the case, then I agree with you. It should definitely be > possible to access environment variables. Otherwise I can't imagine how to > refer to the hostname or the current JAVA path or ... or ... or on each > node?! > > > > Mit freundlichen Grüßen / best regards > > Kay-Uwe Moosheimer > > > > > Am 19.10.2020 um 20:00 schrieb Chad Zobrisky <[email protected]>: > > > > > > Andy, > > > > > > Thanks for the response! > > > > > > When I was thinking through this the deprecation of variables was > > > definitely on my mind but the fact that it already had direct access to > > > environment variables was the simplest path. I think it does make more > > > sense to add access to environment variables to the parameter context, > or > > > allowing a specific scope just for environment variables in the > > > expression language. > > > > > > I think giving access to environment variables actually allows more > > > portability between environments, eg dev, test, prod. Defining those > once > > > and allowing for nifi to pull them in makes sense to me and I think is > > > common in container environments. > > > > > > Looking forward to discussing more and better approaches. > > > Chad > > > > > >> On Mon, Oct 19, 2020 at 1:46 PM Andy LoPresto <[email protected]> > wrote: > > >> > > >> Hi Chad, > > >> > > >> Parameters were introduced as a way to deprecate (NiFi) variables > > >> entirely. I’m not sure that introducing a dependency between the two > is a > > >> positive step forward. I think there is a separate conversation to be > had > > >> about allowing parameters access to environment variables, but I > think this > > >> could introduce problems as parameters are designed for flexibility > and > > >> portability, and moving from a system where a parameter was actually a > > >> pass-through to an environment variable would cause unexpected > problems on > > >> the destination system. > > >> > > >> I think the pros and cons of this need to be clearly enumerated and > > >> discussed here. Thanks for bringing this up. > > >> > > >> > > >> Andy LoPresto > > >> [email protected] > > >> [email protected] > > >> He/Him > > >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > >> > > >>>> On Oct 19, 2020, at 9:43 AM, Chad Zobrisky <[email protected]> > wrote: > > >>> > > >>> Hello, > > >>> > > >>> I was configuring an SSL Context Controller Service today and had the > > >>> keystores and passwords passed into the container via environment > > >>> variables. I thought it would be nice to be able to reference these > from > > >>> the parameter context. Maybe either giving Parameter Context values > the > > >>> VARIABLE_REGISTRY scope in the Expression Language, or a new scope > for > > >>> references external to nifi? > > >>> > > >>> I think for refreshing the Parameter Context on those external > changes, > > >> it > > >>> would require an edit/re-apply just as it does now, and would have to > > >> make > > >>> sure it is well documented. > > >>> > > >>> I'd be interested in creating a PR for this if the idea makes sense > and > > >> is > > >>> acceptable. > > >>> > > >>> Thanks, > > >>> Chad > > >> > > >> > > >
