Hmm - with:
nifi.sensitive.props.algorithm=PBEWITHMD5AND256BITAES-CBC-OPENSSL
I get the following when running "./nifi.sh set-sensitive-properties-key
12characterpassword"
NiFi Properties Processed [/data/1/joeo/nifi-1.14.0/conf/nifi.properties]
Failed to process Flow Configuration [./conf/flow.xml.gz]
org.apache.nifi.encrypt.EncryptionException: Decryption Failed with
Algorithm [PBEWITHMD5AND256BITAES-CBC-OPENSSL]
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:78)
at
org.apache.nifi.flow.encryptor.StandardFlowEncryptor.getOutputEncrypted(StandardFlowEncryptor.java:71)
at
org.apache.nifi.flow.encryptor.StandardFlowEncryptor.lambda$processFlow$0(StandardFlowEncryptor.java:57)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at
java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at
java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
at
org.apache.nifi.flow.encryptor.StandardFlowEncryptor.processFlow(StandardFlowEncryptor.java:54)
at
org.apache.nifi.flow.encryptor.command.SetSensitivePropertiesKey.processFlowConfiguration(SetSensitivePropertiesKey.java:112)
at
org.apache.nifi.flow.encryptor.command.SetSensitivePropertiesKey.run(SetSensitivePropertiesKey.java:97)
at
org.apache.nifi.flow.encryptor.command.SetSensitivePropertiesKey.main(SetSensitivePropertiesKey.java:72)
Caused by: javax.crypto.BadPaddingException: pad block corrupted
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$BufferedGenericBlockCipher.doFinal(Unknown
Source)
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineDoFinal(Unknown
Source)
at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2202)
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:74)
If I change to:
nifi.sensitive.props.algorithm=NIFI_PBKDF2_AES_GCM_256
I get:
NiFi Properties Processed [/data/1/joeo/nifi-1.14.0/conf/nifi.properties]
Failed to process Flow Configuration [./conf/flow.xml.gz]
org.apache.nifi.encrypt.EncryptionException: Decryption Failed with
Algorithm [AES/GCM/NoPadding]
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:78)
at
org.apache.nifi.flow.encryptor.StandardFlowEncryptor.getOutputEncrypted(StandardFlowEncryptor.java:71)
at
org.apache.nifi.flow.encryptor.StandardFlowEncryptor.lambda$processFlow$0(StandardFlowEncryptor.java:57)
at java.base/java.util.Iterator.forEachRemaining(Iterator.java:133)
at
java.base/java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
at
java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658)
at
org.apache.nifi.flow.encryptor.StandardFlowEncryptor.processFlow(StandardFlowEncryptor.java:54)
at
org.apache.nifi.flow.encryptor.command.SetSensitivePropertiesKey.processFlowConfiguration(SetSensitivePropertiesKey.java:112)
at
org.apache.nifi.flow.encryptor.command.SetSensitivePropertiesKey.run(SetSensitivePropertiesKey.java:97)
at
org.apache.nifi.flow.encryptor.command.SetSensitivePropertiesKey.main(SetSensitivePropertiesKey.java:72)
Caused by: javax.crypto.AEADBadTagException: mac check in GCM failed
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$AEADGenericBlockCipher.doFinal(Unknown
Source)
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineDoFinal(Unknown
Source)
at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2202)
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:74)
-Joe
On 7/20/2021 1:39 PM, David Handermann wrote:
Joe,
You're welcome! It looks like there is still a configuration problem
based on the inability to read the flow.xml.gz.
With the existing flow.xml.gz in the conf directory, did you try
running the set-sensitive-properties-key command after setting the PBE
value for the sensitive properties algorithm in nifi.properties?
Regards,
David Handermann
On Tue, Jul 20, 2021 at 12:32 PM Joe Obernberger
<[email protected]> wrote:
Thank you David - your help is great!
I've removed the 1.14.0 node from the cluster and it fires up OK
without a flow file (can access the UI). If I put the flow from
the 1.13.2 version there, I get this error:
2021-07-20 13:31:06,929 WARN [main]
org.apache.nifi.web.server.JettyServer Failed to start web
server... shutting down.
org.apache.nifi.encrypt.EncryptionException: Decryption Failed
with Algorithm [AES/GCM/NoPadding]
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:78)
at
org.apache.nifi.fingerprint.FingerprintFactory.decrypt(FingerprintFactory.java:935)
at
org.apache.nifi.fingerprint.FingerprintFactory.getLoggableRepresentationOfSensitiveValue(FingerprintFactory.java:550)
at
org.apache.nifi.fingerprint.FingerprintFactory.access$200(FingerprintFactory.java:71)
at
org.apache.nifi.fingerprint.FingerprintFactory$6.compare(FingerprintFactory.java:837)
at
org.apache.nifi.fingerprint.FingerprintFactory$6.compare(FingerprintFactory.java:830)
at java.base/java.util.TimSort.binarySort(TimSort.java:296)
at java.base/java.util.TimSort.sort(TimSort.java:239)
at java.base/java.util.Arrays.sort(Arrays.java:1515)
at java.base/java.util.ArrayList.sort(ArrayList.java:1750)
at java.base/java.util.Collections.sort(Collections.java:179)
at
org.apache.nifi.fingerprint.FingerprintFactory.sortElements(FingerprintFactory.java:879)
at
org.apache.nifi.fingerprint.FingerprintFactory.addFlowFileProcessorFingerprint(FingerprintFactory.java:486)
at
org.apache.nifi.fingerprint.FingerprintFactory.addProcessGroupFingerprint(FingerprintFactory.java:368)
at
org.apache.nifi.fingerprint.FingerprintFactory.addProcessGroupFingerprint(FingerprintFactory.java:396)
at
org.apache.nifi.fingerprint.FingerprintFactory.addFlowControllerFingerprint(FingerprintFactory.java:226)
at
org.apache.nifi.fingerprint.FingerprintFactory.createFingerprint(FingerprintFactory.java:168)
at
org.apache.nifi.fingerprint.FingerprintFactory.createFingerprint(FingerprintFactory.java:142)
at
org.apache.nifi.controller.inheritance.FlowFingerprintCheck.checkInheritability(FlowFingerprintCheck.java:45)
at
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:206)
at
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1469)
at
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89)
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810)
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:539)
at
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1068)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
at
org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:997)
at
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
at
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:911)
at
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:423)
at
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:387)
at
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at
org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1129)
at org.apache.nifi.NiFi.<init>(NiFi.java:159)
at org.apache.nifi.NiFi.<init>(NiFi.java:71)
at org.apache.nifi.NiFi.main(NiFi.java:303)
Caused by: javax.crypto.AEADBadTagException: mac check in GCM failed
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$AEADGenericBlockCipher.doFinal(Unknown
Source)
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineDoFinal(Unknown
Source)
at java.base/javax.crypto.Cipher.doFinal(Cipher.java:2202)
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:74)
... 62 common frames omitted
-Joe
On 7/20/2021 1:06 PM, David Handermann wrote:
Joe,
When upgrading from a previous version of NiFi with the default
settings, you should use PBEWITHMD5AND256BITAES-CBC-OPENSSL for
nifi.sensitive.props.algorithm.
If you did not previously have a Sensitive Properties Key
configured for nifi.sensitive.props.key in nifi.properties, you
can run the following command to set a new Sensitive Properties Key.
./bin/nifi.sh set-sensitive-properties-key NewSensitivePropertiesKey
Replace /NewSensitivePropertiesKey /with a randomly generated
string of at least 12 characters.
You will need to run this command on all cluster nodes, using the
same key, to ensure that all nodes share the same configuration.
Please pass along any stack traces if NiFi does not startup after
making those changes.
Regards,
David Handermann
On Tue, Jul 20, 2021 at 11:54 AM Joe Obernberger
<[email protected]> wrote:
Thank you David.
If my prior flow was not encrypted, what do I set the
nifi.sensitive.props.algorithm to? I've tried
NIFI_PBKDF2_AES_GCM_256, and PBEWITHMD5AND256BITAES-CBC-OPENSSL.
-Joe
On 7/20/2021 10:44 AM, David Handermann wrote:
Hi Joe,
Thanks for following up. NiFi supports encryption at
different levels, and always implements some form of
encryption for sensitive processor properties. Using the
previous value for nifi.sensitive.props.algorithm should
allow NiFi 1.14.0 to load the existing flow.xml.gz, but it
sounds like some additional configuration changes are necessary.
All cluster nodes should be running the same version of
NiFi, but if your cluster was not previously configured to
communicate over HTTPS, then you have two options. The
first option is to configure all cluster nodes for HTTP
communication. You should be able to use the nifi.web.http
properties from your current nifi.properties file to
continue running with HTTP on NiFi 1.14.0. The second
option is to configure all cluster nodes for HTTPS
communication. This involves generating or obtaining unique
certificates from a trusted certificate authority for each
cluster node. The current NiFi documentation includes a
guide on secure cluster configuration using the TLS Toolkit:
https://nifi.apache.org/docs/nifi-docs/html/walkthroughs.html#creating-and-securing-a-nifi-cluster-with-the-tls-toolkit
The certificate generated in NiFi 1.14.0 is only suitable
for standalone deployments. As a self-signed certificate, it
is not intended to be used for clustered configurations, so
existing documentation on configuring a secure cluster
provides the recommended approach.
Regards,
David Handermann
On Tue, Jul 20, 2021 at 9:31 AM Joe Obernberger
<[email protected]> wrote:
Thank you David. I tried the new setting, but no go.
I'm sure this is user error on my end; my old flow file
was not encrypted with 1.13.2, but not sure how to bring
it over.
Can Nifi 1.14.x run in the same cluster as 1.13.x?
If I delete the flow file, NiFi runs, but doesn't join
the cluster.
"Failed marshalling 'CONNECTION_REQUEST' protocol
message due to: javax.net.ssl.SSLHandshakeException:
Remote host terminated the handshake"
When I try to connect via a browser (now port 8443), the
browser presents a list of certificates. Where can I
find the 60 day self-signed certificate to import?
-Joe
On 7/19/2021 8:15 PM, David Handermann wrote:
Hi Joe,
Thanks for providing the stack trace associated with
the startup failure. The problem is related to
decryption of sensitive property values stored in the
flow.xml.gz configuration.
Can you provide the value of the following property
from your nifi.properties file?
nifi.sensitive.props.algorithm
In version 1.13.2, the default value was
PBEWITHMD5AND256BITAES-CBC-OPENSSL. In version 1.14.0
the new default value is NIFI_PBKDF2_AES_GCM_256.
Based on the error message, the configured value
appears to be NIFI_PBKDF2_AES_GCM_256, or one of the
other AES_GCM options. However, when upgrading from an
existing flow.xml.gz, this property needs to be the
exact same value used prior to upgrading.
Can you try changing nifi.sensitive.props.algorithm to
PBEWITHMD5AND256BITAES-CBC-OPENSSL?
Regards,
David Handermann
On Mon, Jul 19, 2021 at 6:50 PM Joe Obernberger
<[email protected]> wrote:
Trying to go from 1.13.2 to 1.14.0, but am getting
this error:
2021-07-19 19:47:36,953 WARN [main]
org.apache.nifi.web.server.JettyServer Failed to
start web server...
shutting down.
org.apache.nifi.encrypt.EncryptionException:
Decryption Failed with
Algorithm [AES/GCM/NoPadding]
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:78)
at
org.apache.nifi.fingerprint.FingerprintFactory.decrypt(FingerprintFactory.java:935)
at
org.apache.nifi.fingerprint.FingerprintFactory.getLoggableRepresentationOfSensitiveValue(FingerprintFactory.java:550)
at
org.apache.nifi.fingerprint.FingerprintFactory.access$200(FingerprintFactory.java:71)
at
org.apache.nifi.fingerprint.FingerprintFactory$6.compare(FingerprintFactory.java:837)
at
org.apache.nifi.fingerprint.FingerprintFactory$6.compare(FingerprintFactory.java:830)
at
java.base/java.util.TimSort.binarySort(TimSort.java:296)
at
java.base/java.util.TimSort.sort(TimSort.java:239)
at
java.base/java.util.Arrays.sort(Arrays.java:1515)
at
java.base/java.util.ArrayList.sort(ArrayList.java:1750)
at
java.base/java.util.Collections.sort(Collections.java:179)
at
org.apache.nifi.fingerprint.FingerprintFactory.sortElements(FingerprintFactory.java:879)
at
org.apache.nifi.fingerprint.FingerprintFactory.addFlowFileProcessorFingerprint(FingerprintFactory.java:486)
at
org.apache.nifi.fingerprint.FingerprintFactory.addProcessGroupFingerprint(FingerprintFactory.java:368)
at
org.apache.nifi.fingerprint.FingerprintFactory.addProcessGroupFingerprint(FingerprintFactory.java:396)
at
org.apache.nifi.fingerprint.FingerprintFactory.addFlowControllerFingerprint(FingerprintFactory.java:226)
at
org.apache.nifi.fingerprint.FingerprintFactory.createFingerprint(FingerprintFactory.java:168)
at
org.apache.nifi.fingerprint.FingerprintFactory.createFingerprint(FingerprintFactory.java:142)
at
org.apache.nifi.controller.inheritance.FlowFingerprintCheck.checkInheritability(FlowFingerprintCheck.java:45)
at
org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:206)
at
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1469)
at
org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:89)
at
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:810)
at
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:458)
at
org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1206)
at org.apache.nifi.NiFi.<init>(NiFi.java:159)
at org.apache.nifi.NiFi.<init>(NiFi.java:71)
at org.apache.nifi.NiFi.main(NiFi.java:303)
Caused by: javax.crypto.AEADBadTagException: mac
check in GCM failed
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at
java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher$AEADGenericBlockCipher.doFinal(Unknown
Source)
at
org.bouncycastle.jcajce.provider.symmetric.util.BaseBlockCipher.engineDoFinal(Unknown
Source)
at
java.base/javax.crypto.Cipher.doFinal(Cipher.java:2202)
at
org.apache.nifi.encrypt.CipherPropertyEncryptor.decrypt(CipherPropertyEncryptor.java:74)
... 27 common frames omitted
2021-07-19 19:47:36,953 INFO [Thread-0]
org.apache.nifi.NiFi Initiating
shutdown of Jetty web server...
Any ideas?
Thank you!
-Joe
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
Virus-free. www.avg.com
<http://www.avg.com/email-signature?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=emailclient>
<#m_4246844977010755397_m_1841351089320358696_m_803060838424769034_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
