Re: Using variables in SSLContextService
not > allow the full complement of EL functions to be evaluated. I have verified > this in NiFi 1.9.0-RC2 by putting EL code containing functions into the Input > Directory PD of ListFile, which is scoped to “Variable Registry Only” — it > successfully executes the EL functions. See code in [1] and [2] for more. > I think it is a fair request for the keystore/truststore path property > descriptors of the implementations of SSLContextService and > RestrictedSSLContextService to evaluate EL with the variable scope of VR > only. However, the password properties still will not accept EL at all. I > think there are legitimate discussions to be had around adding the Restricted > component permission to those controller services, and around separating EL > function evaluation from simple variable substitution, but currently those > topics have not been addressed. > > > [1] > https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java#L269 > > <https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java#L269> > [2] > https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/expression/ExpressionLanguageScope.java#L41 > > <https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/expression/ExpressionLanguageScope.java#L41> > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Feb 19, 2019, at 6:16 PM, Beutel, Maximilian >> > <mailto:maximilian.beu...@credit-suisse.com>> wrote: >> >> Andrew, >> >> Yes, I was exploring to see if I can use variable registry values in the >> properties of the SSL Context Service. I wouldn’t need full expression >> language support. >> >> To give an example of what I want to do: >> >> My keystore is a .p12 file, call it store.p12. On my development box it >> resides in a different location than on my prod. >> >> On dev: C:/Users/max/store.p12 >> On prod: /etc/store.p12 >> >> So my idea was to define the keystore path such as >> ${nifi.system.properties.dir}/store.p12 in the CS and then I can easily >> override the directory based on which stage I am using the variable >> registry. >> >> If you guys agree that this is a reasonable request, is it ok for me then to >> raise such a feature request in >> https://issues.apache.org/jira/projects/NIFI/issues/NIFI-4610?filter=allopenissues >> >> <https://issues.apache.org/jira/projects/NIFI/issues/NIFI-4610?filter=allopenissues> >> ? >> >> In the meantime, is there another way to achieve what I’m after? >> >> Thanks! >> >> From: Andrew Grande [mailto:apere...@gmail.com <mailto:apere...@gmail.com>] >> Sent: Wednesday, February 20, 2019 1:53 AM >> To: users@nifi.apache.org <mailto:users@nifi.apache.org> >> Subject: Re: Using variables in SSLContextService >> >> Mike, I think the ask here is for this CS to support variable registry >> values. IIRC, there are other cases in NiFi where EL is not supported, but >> VR is. A fair request, IMO. >> >> Supporting a full EL for the keystore/truststore path is a bad idea, no >> doubt. >> >> Do you agree? >> >> Andrew >> >> On Tue, Feb 19, 2019, 3:33 AM Mike Thomsen > <mailto:mikerthom...@gmail.com>> wrote: >> When expression language is not supported by a field, it won't accept any >> variables. >> >> Mike >> >> On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian >> > <mailto:maximilian.beu...@credit-suisse.com>> wrote: >> Hello! >> >> Also asked the question on IRC, but figured the mailing list might be better >> for this longer question. >> >> For an InvokeHTTP processor I defined a SSL Context Service. In the SSL >> Context Service, in Keystore Filename property, I’d like to use a variable >> which I defined in a nifi.registry file. So in my nifi.properties I have: >> >> nifi.variable.registry.properties=./conf/custom.properties >> >> And in the conf/custom.properties I have: >> >> nifi.system.properties.file=C:/Users/some.file >> >> And in the field Keystore Filename in the SSL Context Service I input: >> >> ${nifi.syst
Re: Using variables in SSLContextService
er=allopenissues > ? > > In the meantime, is there another way to achieve what I’m after? > > Thanks! > > *From:* Andrew Grande [mailto:apere...@gmail.com ] > *Sent:* Wednesday, February 20, 2019 1:53 AM > *To:* users@nifi.apache.org > *Subject:* Re: Using variables in SSLContextService > > Mike, I think the ask here is for this CS to support variable registry > values. IIRC, there are other cases in NiFi where EL is not supported, but > VR is. A fair request, IMO. > > Supporting a full EL for the keystore/truststore path is a bad idea, no > doubt. > > Do you agree? > > > Andrew > On Tue, Feb 19, 2019, 3:33 AM Mike Thomsen wrote: > > When expression language is not supported by a field, it won't accept any > variables. > > Mike > > On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian < > maximilian.beu...@credit-suisse.com> wrote: > > Hello! > > Also asked the question on IRC, but figured the mailing list might be > better for this longer question. > > For an InvokeHTTP processor I defined a SSL Context Service. In the SSL > Context Service, in Keystore Filename property, I’d like to use a variable > which I defined in a nifi.registry file. So in my nifi.properties I have: > > nifi.variable.registry.properties=./conf/custom.properties > > And in the conf/custom.properties I have: > > nifi.system.properties.file=C:/Users/some.file > > And in the field Keystore Filename in the SSL Context Service I input: > > ${nifi.system.properties.file} > > But then saving the SSL Context Service doesn’t work anymore, the > validation fails and says “${nifi.system.properties.file} does not exist”. > The actual file does exist however, but I suspect that the variable doesn’t > get interpolated. > > According to > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.5.0/org.apache.nifi.ssl.StandardSSLContextService/index.html > it seems like expression language is not supported for Keystore Filename > property. Does this also imply that variables won’t work in that field? > > Thanks for your help! > > Max > > > > > == > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > == > > > > > == > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > == > > >
Re: Using variables in SSLContextService
I think there are a couple distinct issues to address here. The claim that allowing EL evaluation for the keystore/truststore path is a security concern. What is the risk here? This input should be trusted (if someone is configuring the SSLContextService, they are aware of & using a system which has a keystore & truststore to which the OS user running NiFi has read access). Any user input which is used to read from the local filesystem anywhere in the application should be validated, but at some point, input from an authenticated and authorized user must be allowed in order to configure the system. One could make the argument that this controller service should be @Restricted, similar to the KeytabCredentialsService used to provide protected access to various Kerberos key tabs without exposing file locations to unauthorized users. I am unclear on the apparent distinction being drawn by some people here between variable substitution and expression language evaluation. My understanding is that a property descriptor can support expression language or not — a boolean decision. _If_ it supports expression language, it can allow variable access to “only" the Variable Registry (which includes custom properties files and OS environment variables) or it can include the flowfile attributes of each flowfile that passes through the component. Some properties are scoped to not allow per-flowfile access, but I am unaware of any property descriptor which supports variable substitution which does not allow the full complement of EL functions to be evaluated. I have verified this in NiFi 1.9.0-RC2 by putting EL code containing functions into the Input Directory PD of ListFile, which is scoped to “Variable Registry Only” — it successfully executes the EL functions. See code in [1] and [2] for more. I think it is a fair request for the keystore/truststore path property descriptors of the implementations of SSLContextService and RestrictedSSLContextService to evaluate EL with the variable scope of VR only. However, the password properties still will not accept EL at all. I think there are legitimate discussions to be had around adding the Restricted component permission to those controller services, and around separating EL function evaluation from simple variable substitution, but currently those topics have not been addressed. [1] https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/components/PropertyDescriptor.java#L269 [2] https://github.com/apache/nifi/blob/master/nifi-api/src/main/java/org/apache/nifi/expression/ExpressionLanguageScope.java#L41 Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Feb 19, 2019, at 6:16 PM, Beutel, Maximilian > wrote: > > Andrew, > > Yes, I was exploring to see if I can use variable registry values in the > properties of the SSL Context Service. I wouldn’t need full expression > language support. > > To give an example of what I want to do: > > My keystore is a .p12 file, call it store.p12. On my development box it > resides in a different location than on my prod. > > On dev: C:/Users/max/store.p12 > On prod: /etc/store.p12 > > So my idea was to define the keystore path such as > ${nifi.system.properties.dir}/store.p12 in the CS and then I can easily > override the directory based on which stage I am using the variable registry. > > If you guys agree that this is a reasonable request, is it ok for me then to > raise such a feature request in > https://issues.apache.org/jira/projects/NIFI/issues/NIFI-4610?filter=allopenissues > > <https://issues.apache.org/jira/projects/NIFI/issues/NIFI-4610?filter=allopenissues> > ? > > In the meantime, is there another way to achieve what I’m after? > > Thanks! > > From: Andrew Grande [mailto:apere...@gmail.com] > Sent: Wednesday, February 20, 2019 1:53 AM > To: users@nifi.apache.org > Subject: Re: Using variables in SSLContextService > > Mike, I think the ask here is for this CS to support variable registry > values. IIRC, there are other cases in NiFi where EL is not supported, but VR > is. A fair request, IMO. > > Supporting a full EL for the keystore/truststore path is a bad idea, no doubt. > > Do you agree? > > Andrew > > On Tue, Feb 19, 2019, 3:33 AM Mike Thomsen <mailto:mikerthom...@gmail.com>> wrote: > When expression language is not supported by a field, it won't accept any > variables. > > Mike > > On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian > <mailto:maximilian.beu...@credit-suisse.com>> wrote: > Hello! > > Also asked the question on IRC, but figured the mailing list might be better > for this longer
Re: Using variables in SSLContextService
I agree that there is value in having EL enabled properties for some of the SSLContext properties. I dont understand the security concern raised but am open to what I might be missing. It would need variable and env var access. Thanks Joe On Tue, Feb 19, 2019 at 9:16 PM Beutel, Maximilian < maximilian.beu...@credit-suisse.com> wrote: > Andrew, > > > > Yes, I was exploring to see if I can use variable registry values in the > properties of the SSL Context Service. I wouldn’t need full expression > language support. > > > > To give an example of what I want to do: > > > > My keystore is a .p12 file, call it store.p12. On my development box it > resides in a different location than on my prod. > > > > On dev: C:/Users/max/store.p12 > > On prod: /etc/store.p12 > > > > So my idea was to define the keystore path such as > ${nifi.system.properties.dir}/store.p12 in the CS and then I can easily > override the directory based on which stage I am using the variable > registry. > > > > If you guys agree that this is a reasonable request, is it ok for me then > to raise such a feature request in > https://issues.apache.org/jira/projects/NIFI/issues/NIFI-4610?filter=allopenissues > ? > > > > In the meantime, is there another way to achieve what I’m after? > > > > Thanks! > > > > *From:* Andrew Grande [mailto:apere...@gmail.com] > *Sent:* Wednesday, February 20, 2019 1:53 AM > *To:* users@nifi.apache.org > *Subject:* Re: Using variables in SSLContextService > > > > Mike, I think the ask here is for this CS to support variable registry > values. IIRC, there are other cases in NiFi where EL is not supported, but > VR is. A fair request, IMO. > > > > Supporting a full EL for the keystore/truststore path is a bad idea, no > doubt. > > > > Do you agree? > > > > Andrew > > On Tue, Feb 19, 2019, 3:33 AM Mike Thomsen wrote: > > When expression language is not supported by a field, it won't accept any > variables. > > > > Mike > > > > On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian < > maximilian.beu...@credit-suisse.com> wrote: > > Hello! > > > > Also asked the question on IRC, but figured the mailing list might be > better for this longer question. > > > > For an InvokeHTTP processor I defined a SSL Context Service. In the SSL > Context Service, in Keystore Filename property, I’d like to use a variable > which I defined in a nifi.registry file. So in my nifi.properties I have: > > > > nifi.variable.registry.properties=./conf/custom.properties > > > > And in the conf/custom.properties I have: > > > > nifi.system.properties.file=C:/Users/some.file > > > > And in the field Keystore Filename in the SSL Context Service I input: > > > > ${nifi.system.properties.file} > > > > But then saving the SSL Context Service doesn’t work anymore, the > validation fails and says “${nifi.system.properties.file} does not exist”. > The actual file does exist however, but I suspect that the variable doesn’t > get interpolated. > > > > According to > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.5.0/org.apache.nifi.ssl.StandardSSLContextService/index.html > it seems like expression language is not supported for Keystore Filename > property. Does this also imply that variables won’t work in that field? > > > > Thanks for your help! > > > > Max > > > > > > > == > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > == > > > > > == > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > == >
RE: Using variables in SSLContextService
Andrew, Yes, I was exploring to see if I can use variable registry values in the properties of the SSL Context Service. I wouldn’t need full expression language support. To give an example of what I want to do: My keystore is a .p12 file, call it store.p12. On my development box it resides in a different location than on my prod. On dev: C:/Users/max/store.p12 On prod: /etc/store.p12 So my idea was to define the keystore path such as ${nifi.system.properties.dir}/store.p12 in the CS and then I can easily override the directory based on which stage I am using the variable registry. If you guys agree that this is a reasonable request, is it ok for me then to raise such a feature request in https://issues.apache.org/jira/projects/NIFI/issues/NIFI-4610?filter=allopenissues ? In the meantime, is there another way to achieve what I’m after? Thanks! From: Andrew Grande [mailto:apere...@gmail.com] Sent: Wednesday, February 20, 2019 1:53 AM To: users@nifi.apache.org Subject: Re: Using variables in SSLContextService Mike, I think the ask here is for this CS to support variable registry values. IIRC, there are other cases in NiFi where EL is not supported, but VR is. A fair request, IMO. Supporting a full EL for the keystore/truststore path is a bad idea, no doubt. Do you agree? Andrew On Tue, Feb 19, 2019, 3:33 AM Mike Thomsen mailto:mikerthom...@gmail.com>> wrote: When expression language is not supported by a field, it won't accept any variables. Mike On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian mailto:maximilian.beu...@credit-suisse.com>> wrote: Hello! Also asked the question on IRC, but figured the mailing list might be better for this longer question. For an InvokeHTTP processor I defined a SSL Context Service. In the SSL Context Service, in Keystore Filename property, I’d like to use a variable which I defined in a nifi.registry file. So in my nifi.properties I have: nifi.variable.registry.properties=./conf/custom.properties And in the conf/custom.properties I have: nifi.system.properties.file=C:/Users/some.file And in the field Keystore Filename in the SSL Context Service I input: ${nifi.system.properties.file} But then saving the SSL Context Service doesn’t work anymore, the validation fails and says “${nifi.system.properties.file} does not exist”. The actual file does exist however, but I suspect that the variable doesn’t get interpolated. According to https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.5.0/org.apache.nifi.ssl.StandardSSLContextService/index.html it seems like expression language is not supported for Keystore Filename property. Does this also imply that variables won’t work in that field? Thanks for your help! Max == Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html == === Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ===
Re: Using variables in SSLContextService
Yeah, I think adding a ticket for VR-level EL is perfectly reasonable. On Tue, Feb 19, 2019 at 12:52 PM Andrew Grande wrote: > Mike, I think the ask here is for this CS to support variable registry > values. IIRC, there are other cases in NiFi where EL is not supported, but > VR is. A fair request, IMO. > > Supporting a full EL for the keystore/truststore path is a bad idea, no > doubt. > > Do you agree? > > Andrew > > On Tue, Feb 19, 2019, 3:33 AM Mike Thomsen wrote: > >> When expression language is not supported by a field, it won't accept any >> variables. >> >> Mike >> >> On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian < >> maximilian.beu...@credit-suisse.com> wrote: >> >>> Hello! >>> >>> >>> >>> Also asked the question on IRC, but figured the mailing list might be >>> better for this longer question. >>> >>> >>> >>> For an InvokeHTTP processor I defined a SSL Context Service. In the SSL >>> Context Service, in Keystore Filename property, I’d like to use a variable >>> which I defined in a nifi.registry file. So in my nifi.properties I have: >>> >>> >>> >>> nifi.variable.registry.properties=./conf/custom.properties >>> >>> >>> >>> And in the conf/custom.properties I have: >>> >>> >>> >>> nifi.system.properties.file=C:/Users/some.file >>> >>> >>> >>> And in the field Keystore Filename in the SSL Context Service I input: >>> >>> >>> >>> ${nifi.system.properties.file} >>> >>> >>> >>> But then saving the SSL Context Service doesn’t work anymore, the >>> validation fails and says “${nifi.system.properties.file} does not exist”. >>> The actual file does exist however, but I suspect that the variable doesn’t >>> get interpolated. >>> >>> >>> >>> According to >>> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.5.0/org.apache.nifi.ssl.StandardSSLContextService/index.html >>> it seems like expression language is not supported for Keystore Filename >>> property. Does this also imply that variables won’t work in that field? >>> >>> >>> >>> Thanks for your help! >>> >>> >>> >>> Max >>> >>> >>> >>> >>> >>> == >>> Please access the attached hyperlink for an important electronic >>> communications disclaimer: >>> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html >>> >>> == >>> >>
Re: Using variables in SSLContextService
Mike, I think the ask here is for this CS to support variable registry values. IIRC, there are other cases in NiFi where EL is not supported, but VR is. A fair request, IMO. Supporting a full EL for the keystore/truststore path is a bad idea, no doubt. Do you agree? Andrew On Tue, Feb 19, 2019, 3:33 AM Mike Thomsen wrote: > When expression language is not supported by a field, it won't accept any > variables. > > Mike > > On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian < > maximilian.beu...@credit-suisse.com> wrote: > >> Hello! >> >> >> >> Also asked the question on IRC, but figured the mailing list might be >> better for this longer question. >> >> >> >> For an InvokeHTTP processor I defined a SSL Context Service. In the SSL >> Context Service, in Keystore Filename property, I’d like to use a variable >> which I defined in a nifi.registry file. So in my nifi.properties I have: >> >> >> >> nifi.variable.registry.properties=./conf/custom.properties >> >> >> >> And in the conf/custom.properties I have: >> >> >> >> nifi.system.properties.file=C:/Users/some.file >> >> >> >> And in the field Keystore Filename in the SSL Context Service I input: >> >> >> >> ${nifi.system.properties.file} >> >> >> >> But then saving the SSL Context Service doesn’t work anymore, the >> validation fails and says “${nifi.system.properties.file} does not exist”. >> The actual file does exist however, but I suspect that the variable doesn’t >> get interpolated. >> >> >> >> According to >> https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.5.0/org.apache.nifi.ssl.StandardSSLContextService/index.html >> it seems like expression language is not supported for Keystore Filename >> property. Does this also imply that variables won’t work in that field? >> >> >> >> Thanks for your help! >> >> >> >> Max >> >> >> >> >> >> == >> Please access the attached hyperlink for an important electronic >> communications disclaimer: >> http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html >> >> == >> >
Re: Using variables in SSLContextService
When expression language is not supported by a field, it won't accept any variables. Mike On Mon, Feb 18, 2019 at 10:34 PM Beutel, Maximilian < maximilian.beu...@credit-suisse.com> wrote: > Hello! > > > > Also asked the question on IRC, but figured the mailing list might be > better for this longer question. > > > > For an InvokeHTTP processor I defined a SSL Context Service. In the SSL > Context Service, in Keystore Filename property, I’d like to use a variable > which I defined in a nifi.registry file. So in my nifi.properties I have: > > > > nifi.variable.registry.properties=./conf/custom.properties > > > > And in the conf/custom.properties I have: > > > > nifi.system.properties.file=C:/Users/some.file > > > > And in the field Keystore Filename in the SSL Context Service I input: > > > > ${nifi.system.properties.file} > > > > But then saving the SSL Context Service doesn’t work anymore, the > validation fails and says “${nifi.system.properties.file} does not exist”. > The actual file does exist however, but I suspect that the variable doesn’t > get interpolated. > > > > According to > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.5.0/org.apache.nifi.ssl.StandardSSLContextService/index.html > it seems like expression language is not supported for Keystore Filename > property. Does this also imply that variables won’t work in that field? > > > > Thanks for your help! > > > > Max > > > > > > == > Please access the attached hyperlink for an important electronic > communications disclaimer: > http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html > > == >
Using variables in SSLContextService
Hello! Also asked the question on IRC, but figured the mailing list might be better for this longer question. For an InvokeHTTP processor I defined a SSL Context Service. In the SSL Context Service, in Keystore Filename property, I'd like to use a variable which I defined in a nifi.registry file. So in my nifi.properties I have: nifi.variable.registry.properties=./conf/custom.properties And in the conf/custom.properties I have: nifi.system.properties.file=C:/Users/some.file And in the field Keystore Filename in the SSL Context Service I input: ${nifi.system.properties.file} But then saving the SSL Context Service doesn't work anymore, the validation fails and says "${nifi.system.properties.file} does not exist". The actual file does exist however, but I suspect that the variable doesn't get interpolated. According to https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-ssl-context-service-nar/1.5.0/org.apache.nifi.ssl.StandardSSLContextService/index.html it seems like expression language is not supported for Keystore Filename property. Does this also imply that variables won't work in that field? Thanks for your help! Max === Please access the attached hyperlink for an important electronic communications disclaimer: http://www.credit-suisse.com/legal/en/disclaimer_email_ib.html ===