Re: SSL_ERROR_NO_CYPHER_OVERLAP
No worries Andy, I have another issue - I'm sure it's a much simpler one, but it has me stumped. I still get the insufficient permissions error, even after adding myself to the authorizers.xml file as per the admin guide. I added the "EMAILADDRESS=" part of the subject after receiving an unkown user error with that as part of the DN (checking the certificate and the subject contains the email address part). Output from nifi-user.log 2018-05-05 14:10:29,034 INFO [NiFi Web Server-21] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: Kerberos ticket login not supported by this NiFi.. Returning Conflict response. 2018-05-05 14:10:29,249 INFO [NiFi Web Server-19] o.a.n.w.a.c.IllegalStateExceptionMapper java.lang.IllegalStateException: OpenId Connect is not configured.. Returning Conflict response. 2018-05-05 14:10:29,310 INFO [NiFi Web Server-76] o.a.n.w.s.NiFiAuthenticationFilter Attempting request for (EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah) GET https://nifi1.blah/nifi-api/flow/current-user (source ip: 192.168.10.192) 2018-05-05 14:10:29,310 INFO [NiFi Web Server-76] o.a.n.w.s.NiFiAuthenticationFilter Authentication success for EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah 2018-05-05 14:10:29,312 INFO [NiFi Web Server-76] o.a.n.w.a.c.AccessDeniedExceptionMapper identity[EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah], groups[] does not have permission to access the requested resource. Unable to view the user interface. Returning Forbidden response. Partial authorizers.xml (the only changes I have made from the install are in red) file-user-group-provider org.apache.nifi.authorization.FileUserGroupProvider ./conf/users.xml EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah file-access-policy-provider org.apache.nifi.authorization.FileAccessPolicyProvider file-user-group-provider ./conf/authorizations.xml EMAILADDRESS=phil@blah, CN=Phil H, OU=Blah On Sat, May 5, 2018 at 11:35 AM, Andy LoPrestowrote: > Phil, > > Thanks for documenting the steps you took to get it working. This will > definitely be helpful to anyone who has a similar problem in the future. I > suspected it was the missing private key and I forgot to explain that yes, > to import a PEM-encoded public key and private key into a JKS keystore, you > do have to go through the PKCS12 intermediary. > > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On May 4, 2018, at 4:36 PM, Phil H wrote: > > Sorry for talking to myself so much. I have solved the problem! > > Once I realised that I had never supplied the private key to keytool when I > imported the certificate, I started googling along those lines. > > I found this link from 2008 which provided the solution > http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html > > Basically you use OpenSSL to create a PKCS12 file from the cert and key, > and then keytool to import that (as an existing keystore, not as a > certificate) > > From there, everything else just worked (including client authentication > through the browser). Now I have the "Insufficient permissions" problem, > which there is loads of documentation as to how to move forwards. > > Thanks! > Phil > > On Sat, May 5, 2018 at 8:27 AM, Phil H wrote: > > Further info > > In the httpd installation, I need the private key for the certificate and > it's paraphrase. That private key has not played a part of the NiFi install > thus far (seems like an important thing - the "keystore" only has a > certificate, not a key) > > The reason why I used PKCS12 for the keystore was some Jetty debug output > telling me that was preferred to JKS. Interestingly it gave me contrary > advice for the trust store. > > On Sat, 5 May 2018 at 08:22, Phil H wrote: > > Hi Andy, > > Sorry - poor use of words (it was late when I wrote the email). When I > said "client" certificate I meant the certificate for the NiFi server (no > idea why I wrote "client") > > I'm not trying to use certificate authentication yet - simply to get NiFi > operating over TLS. > > When I test with OpenSSL, I get the same Nifi server log output as seen > with a connection attempt from a browser. > > On the Cipher issue, when I use OpenSSL to connect using said certificate > using httpd, it lists the successful cipher as one of those ignored by > Jetty, albeit with less underscores in the onscreen output: > TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 > > All machines involved are CentOS 7 - the only add-ons being Java 1.8 > (current release) and NiFi 1.6.0 (also current release) > > On Sat, 5 May 2018 at 01:26, Andy LoPresto wrote: > > Hi Phil, > > Sorry to hear you are having this problem. I have a couple steps you
Re: SSL_ERROR_NO_CYPHER_OVERLAP
Phil, Thanks for documenting the steps you took to get it working. This will definitely be helpful to anyone who has a similar problem in the future. I suspected it was the missing private key and I forgot to explain that yes, to import a PEM-encoded public key and private key into a JKS keystore, you do have to go through the PKCS12 intermediary. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On May 4, 2018, at 4:36 PM, Phil Hwrote: > > Sorry for talking to myself so much. I have solved the problem! > > Once I realised that I had never supplied the private key to keytool when I > imported the certificate, I started googling along those lines. > > I found this link from 2008 which provided the solution > http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html > > Basically you use OpenSSL to create a PKCS12 file from the cert and key, > and then keytool to import that (as an existing keystore, not as a > certificate) > > From there, everything else just worked (including client authentication > through the browser). Now I have the "Insufficient permissions" problem, > which there is loads of documentation as to how to move forwards. > > Thanks! > Phil > > On Sat, May 5, 2018 at 8:27 AM, Phil H wrote: > >> Further info >> >> In the httpd installation, I need the private key for the certificate and >> it's paraphrase. That private key has not played a part of the NiFi install >> thus far (seems like an important thing - the "keystore" only has a >> certificate, not a key) >> >> The reason why I used PKCS12 for the keystore was some Jetty debug output >> telling me that was preferred to JKS. Interestingly it gave me contrary >> advice for the trust store. >> >> On Sat, 5 May 2018 at 08:22, Phil H wrote: >> >>> Hi Andy, >>> >>> Sorry - poor use of words (it was late when I wrote the email). When I >>> said "client" certificate I meant the certificate for the NiFi server (no >>> idea why I wrote "client") >>> >>> I'm not trying to use certificate authentication yet - simply to get NiFi >>> operating over TLS. >>> >>> When I test with OpenSSL, I get the same Nifi server log output as seen >>> with a connection attempt from a browser. >>> >>> On the Cipher issue, when I use OpenSSL to connect using said certificate >>> using httpd, it lists the successful cipher as one of those ignored by >>> Jetty, albeit with less underscores in the onscreen output: >>> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 >>> >>> All machines involved are CentOS 7 - the only add-ons being Java 1.8 >>> (current release) and NiFi 1.6.0 (also current release) >>> >>> On Sat, 5 May 2018 at 01:26, Andy LoPresto wrote: >>> Hi Phil, Sorry to hear you are having this problem. I have a couple steps you can try to resolve this. First, to clarify the terminology for NiFi, a “client certificate” refers to a public certificate and private key which in combination allow a client to uniquely identify itself and authenticate on a mutual authentication TLS connection. In NiFi terminology, the client certificate identifies a user or service which connects to NiFi. The “server certificate” identifies the NiFi service, and the CA is what signs one (or both) of those certificates. The “no cipher suites in common” error can occur when there are legitimately no cipher suites that both the client and server support. This can be verified by using the OpenSSL s_client tool to make a connection from the client to the server. I’ve pasted a sample invocation below. $ openssl s_client -connect -debug -state -cert -key -CAfile However, that error can also appear when the keystore does not contain a valid private key to be used. I suspect the keystore you generated for NiFi does not have the private key. You can verify this by examining the nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output. If you do not see this, you have only the public certificate in the file. Importing that into a keystore means that NiFi (or any other service using that keystore) will not be able to sign or decrypt any information encrypted with the public key, so it won’t be able to support any cipher suites that rely on RSA encryption or signatures. The nifi1.crt you imported into the keystore may also not have the complete certificate chain encoded, in which case when the server presents that certificate on an incoming connection, the client (command-line or browser) won’t be able to verify and trust it. You’ll get a different error, but it is something to be aware of. Is there a
Re: SSL_ERROR_NO_CYPHER_OVERLAP
Sorry for talking to myself so much. I have solved the problem! Once I realised that I had never supplied the private key to keytool when I imported the certificate, I started googling along those lines. I found this link from 2008 which provided the solution http://cunning.sharp.fm/2008/06/importing_private_keys_into_a.html Basically you use OpenSSL to create a PKCS12 file from the cert and key, and then keytool to import that (as an existing keystore, not as a certificate) >From there, everything else just worked (including client authentication through the browser). Now I have the "Insufficient permissions" problem, which there is loads of documentation as to how to move forwards. Thanks! Phil On Sat, May 5, 2018 at 8:27 AM, Phil Hwrote: > Further info > > In the httpd installation, I need the private key for the certificate and > it's paraphrase. That private key has not played a part of the NiFi install > thus far (seems like an important thing - the "keystore" only has a > certificate, not a key) > > The reason why I used PKCS12 for the keystore was some Jetty debug output > telling me that was preferred to JKS. Interestingly it gave me contrary > advice for the trust store. > > On Sat, 5 May 2018 at 08:22, Phil H wrote: > >> Hi Andy, >> >> Sorry - poor use of words (it was late when I wrote the email). When I >> said "client" certificate I meant the certificate for the NiFi server (no >> idea why I wrote "client") >> >> I'm not trying to use certificate authentication yet - simply to get NiFi >> operating over TLS. >> >> When I test with OpenSSL, I get the same Nifi server log output as seen >> with a connection attempt from a browser. >> >> On the Cipher issue, when I use OpenSSL to connect using said certificate >> using httpd, it lists the successful cipher as one of those ignored by >> Jetty, albeit with less underscores in the onscreen output: >> TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 >> >> All machines involved are CentOS 7 - the only add-ons being Java 1.8 >> (current release) and NiFi 1.6.0 (also current release) >> >> On Sat, 5 May 2018 at 01:26, Andy LoPresto wrote: >> >>> Hi Phil, >>> >>> Sorry to hear you are having this problem. I have a couple steps you can >>> try to resolve this. >>> >>> First, to clarify the terminology for NiFi, a “client certificate” >>> refers to a public certificate and private key which in combination allow a >>> client to uniquely identify itself and authenticate on a mutual >>> authentication TLS connection. In NiFi terminology, the client certificate >>> identifies a user or service which connects to NiFi. The “server >>> certificate” identifies the NiFi service, and the CA is what signs one (or >>> both) of those certificates. >>> >>> The “no cipher suites in common” error can occur when there are >>> legitimately no cipher suites that both the client and server support. This >>> can be verified by using the OpenSSL s_client tool to make a connection >>> from the client to the server. I’ve pasted a sample invocation below. >>> >>> $ openssl s_client -connect -debug -state -cert >>> -key -CAfile >>> >>> >>> However, that error can also appear when the keystore does not contain a >>> valid private key to be used. I suspect the keystore you generated for NiFi >>> does not have the private key. You can verify this by examining the >>> nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see >>> a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output. >>> If you do not see this, you have only the public certificate in the file. >>> Importing that into a keystore means that NiFi (or any other service using >>> that keystore) will not be able to sign or decrypt any information >>> encrypted with the public key, so it won’t be able to support any cipher >>> suites that rely on RSA encryption or signatures. >>> >>> The nifi1.crt you imported into the keystore may also not have the >>> complete certificate chain encoded, in which case when the server presents >>> that certificate on an incoming connection, the client (command-line or >>> browser) won’t be able to verify and trust it. You’ll get a different >>> error, but it is something to be aware of. >>> >>> Is there a reason you chose to use a PKCS12 keystore in this scenario? >>> Usually we recommend using JKS for both the keystore and the truststore. >>> >>> I hope this helps. If none of this resolves your issues, please let us >>> know and we can continue to help. >>> >>> Andy LoPresto >>> alopre...@apache.org >>> *alopresto.apa...@gmail.com * >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>> On May 4, 2018, at 4:03 AM, Phil H wrote: >>> >>> Hi all, >>> >>> I am trying to secure my NiFi installation. I have a client certificate >>> (nifi1.crt) and the CA for the intranet (ca.crt). I created the trust >>> and >>>
Re: SSL_ERROR_NO_CYPHER_OVERLAP
Further info In the httpd installation, I need the private key for the certificate and it's paraphrase. That private key has not played a part of the NiFi install thus far (seems like an important thing - the "keystore" only has a certificate, not a key) The reason why I used PKCS12 for the keystore was some Jetty debug output telling me that was preferred to JKS. Interestingly it gave me contrary advice for the trust store. On Sat, 5 May 2018 at 08:22, Phil Hwrote: > Hi Andy, > > Sorry - poor use of words (it was late when I wrote the email). When I > said "client" certificate I meant the certificate for the NiFi server (no > idea why I wrote "client") > > I'm not trying to use certificate authentication yet - simply to get NiFi > operating over TLS. > > When I test with OpenSSL, I get the same Nifi server log output as seen > with a connection attempt from a browser. > > On the Cipher issue, when I use OpenSSL to connect using said certificate > using httpd, it lists the successful cipher as one of those ignored by > Jetty, albeit with less underscores in the onscreen output: > TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 > > All machines involved are CentOS 7 - the only add-ons being Java 1.8 > (current release) and NiFi 1.6.0 (also current release) > > On Sat, 5 May 2018 at 01:26, Andy LoPresto wrote: > >> Hi Phil, >> >> Sorry to hear you are having this problem. I have a couple steps you can >> try to resolve this. >> >> First, to clarify the terminology for NiFi, a “client certificate” refers >> to a public certificate and private key which in combination allow a client >> to uniquely identify itself and authenticate on a mutual authentication TLS >> connection. In NiFi terminology, the client certificate identifies a user >> or service which connects to NiFi. The “server certificate” identifies the >> NiFi service, and the CA is what signs one (or both) of those certificates. >> >> The “no cipher suites in common” error can occur when there are >> legitimately no cipher suites that both the client and server support. This >> can be verified by using the OpenSSL s_client tool to make a connection >> from the client to the server. I’ve pasted a sample invocation below. >> >> $ openssl s_client -connect -debug -state -cert >> -key -CAfile >> >> >> However, that error can also appear when the keystore does not contain a >> valid private key to be used. I suspect the keystore you generated for NiFi >> does not have the private key. You can verify this by examining the >> nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see >> a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output. >> If you do not see this, you have only the public certificate in the file. >> Importing that into a keystore means that NiFi (or any other service using >> that keystore) will not be able to sign or decrypt any information >> encrypted with the public key, so it won’t be able to support any cipher >> suites that rely on RSA encryption or signatures. >> >> The nifi1.crt you imported into the keystore may also not have the >> complete certificate chain encoded, in which case when the server presents >> that certificate on an incoming connection, the client (command-line or >> browser) won’t be able to verify and trust it. You’ll get a different >> error, but it is something to be aware of. >> >> Is there a reason you chose to use a PKCS12 keystore in this scenario? >> Usually we recommend using JKS for both the keystore and the truststore. >> >> I hope this helps. If none of this resolves your issues, please let us >> know and we can continue to help. >> >> Andy LoPresto >> alopre...@apache.org >> *alopresto.apa...@gmail.com * >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >> On May 4, 2018, at 4:03 AM, Phil H wrote: >> >> Hi all, >> >> I am trying to secure my NiFi installation. I have a client certificate >> (nifi1.crt) and the CA for the intranet (ca.crt). I created the trust and >> keystores as below: >> >> keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore >> server_keystore.p12 -storetype PKCS12 >> >> keytool -import -file ca.crt -alias cacert -keystore truststore.jks >> >> And the relevant nifi.properties are set as follows >> >> nifi.security.keystore=./conf/server_keystore.p12 >> nifi.security.keystoreType=PKCS12 >> nifi.security.keystorePasswd= >> nifi.security.keyPasswd= >> nifi.security.truststore=./conf/truststore.jks >> nifi.security.truststoreType=JKS >> nifi.security.truststorePasswd= >> >> When I try and access the site via https, I receive the above error in >> Firefox, and the following in the nifi-bootstrap.log (I have enabled >> additional debugging). >> >> Using both of these certificates inside Apache httpd works on the client >> as >> expected, so the certificates are fine. I have seen some references to >> bugs/features
Re: SSL_ERROR_NO_CYPHER_OVERLAP
Hi Andy, Sorry - poor use of words (it was late when I wrote the email). When I said "client" certificate I meant the certificate for the NiFi server (no idea why I wrote "client") I'm not trying to use certificate authentication yet - simply to get NiFi operating over TLS. When I test with OpenSSL, I get the same Nifi server log output as seen with a connection attempt from a browser. On the Cipher issue, when I use OpenSSL to connect using said certificate using httpd, it lists the successful cipher as one of those ignored by Jetty, albeit with less underscores in the onscreen output: TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 All machines involved are CentOS 7 - the only add-ons being Java 1.8 (current release) and NiFi 1.6.0 (also current release) On Sat, 5 May 2018 at 01:26, Andy LoPrestowrote: > Hi Phil, > > Sorry to hear you are having this problem. I have a couple steps you can > try to resolve this. > > First, to clarify the terminology for NiFi, a “client certificate” refers > to a public certificate and private key which in combination allow a client > to uniquely identify itself and authenticate on a mutual authentication TLS > connection. In NiFi terminology, the client certificate identifies a user > or service which connects to NiFi. The “server certificate” identifies the > NiFi service, and the CA is what signs one (or both) of those certificates. > > The “no cipher suites in common” error can occur when there are > legitimately no cipher suites that both the client and server support. This > can be verified by using the OpenSSL s_client tool to make a connection > from the client to the server. I’ve pasted a sample invocation below. > > $ openssl s_client -connect -debug -state -cert > -key -CAfile > > > However, that error can also appear when the keystore does not contain a > valid private key to be used. I suspect the keystore you generated for NiFi > does not have the private key. You can verify this by examining the > nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see > a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output. > If you do not see this, you have only the public certificate in the file. > Importing that into a keystore means that NiFi (or any other service using > that keystore) will not be able to sign or decrypt any information > encrypted with the public key, so it won’t be able to support any cipher > suites that rely on RSA encryption or signatures. > > The nifi1.crt you imported into the keystore may also not have the > complete certificate chain encoded, in which case when the server presents > that certificate on an incoming connection, the client (command-line or > browser) won’t be able to verify and trust it. You’ll get a different > error, but it is something to be aware of. > > Is there a reason you chose to use a PKCS12 keystore in this scenario? > Usually we recommend using JKS for both the keystore and the truststore. > > I hope this helps. If none of this resolves your issues, please let us > know and we can continue to help. > > Andy LoPresto > alopre...@apache.org > *alopresto.apa...@gmail.com * > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > On May 4, 2018, at 4:03 AM, Phil H wrote: > > Hi all, > > I am trying to secure my NiFi installation. I have a client certificate > (nifi1.crt) and the CA for the intranet (ca.crt). I created the trust and > keystores as below: > > keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore > server_keystore.p12 -storetype PKCS12 > > keytool -import -file ca.crt -alias cacert -keystore truststore.jks > > And the relevant nifi.properties are set as follows > > nifi.security.keystore=./conf/server_keystore.p12 > nifi.security.keystoreType=PKCS12 > nifi.security.keystorePasswd= > nifi.security.keyPasswd= > nifi.security.truststore=./conf/truststore.jks > nifi.security.truststoreType=JKS > nifi.security.truststorePasswd= > > When I try and access the site via https, I receive the above error in > Firefox, and the following in the nifi-bootstrap.log (I have enabled > additional debugging). > > Using both of these certificates inside Apache httpd works on the client as > expected, so the certificates are fine. I have seen some references to > bugs/features in Jetty under Java 1.8 related to older TLS versions, but > I'm at a loss to explain this! Help!! > > Thanks, > Phil > > 2018-05-04 20:57:17,406 INFO [NiFi logging handler] org.apache.nifi.StdOut > Using SSLEngineImpl. > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Allow unsafe renegotiation: false > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Allow legacy hello messages: true > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Is initial handshake: true > 2018-05-04 20:57:17,407 INFO [NiFi logging handler]
Re: SSL_ERROR_NO_CYPHER_OVERLAP
Hi Phil, Sorry to hear you are having this problem. I have a couple steps you can try to resolve this. First, to clarify the terminology for NiFi, a “client certificate” refers to a public certificate and private key which in combination allow a client to uniquely identify itself and authenticate on a mutual authentication TLS connection. In NiFi terminology, the client certificate identifies a user or service which connects to NiFi. The “server certificate” identifies the NiFi service, and the CA is what signs one (or both) of those certificates. The “no cipher suites in common” error can occur when there are legitimately no cipher suites that both the client and server support. This can be verified by using the OpenSSL s_client tool to make a connection from the client to the server. I’ve pasted a sample invocation below. $ openssl s_client -connect -debug -state -cert -key -CAfile However, that error can also appear when the keystore does not contain a valid private key to be used. I suspect the keystore you generated for NiFi does not have the private key. You can verify this by examining the nifi1.crt file you imported. If you run `$ more nifi1.crt`, you should see a line “-BEGIN PRIVATE KEY-“ and then some Base64-encoded output. If you do not see this, you have only the public certificate in the file. Importing that into a keystore means that NiFi (or any other service using that keystore) will not be able to sign or decrypt any information encrypted with the public key, so it won’t be able to support any cipher suites that rely on RSA encryption or signatures. The nifi1.crt you imported into the keystore may also not have the complete certificate chain encoded, in which case when the server presents that certificate on an incoming connection, the client (command-line or browser) won’t be able to verify and trust it. You’ll get a different error, but it is something to be aware of. Is there a reason you chose to use a PKCS12 keystore in this scenario? Usually we recommend using JKS for both the keystore and the truststore. I hope this helps. If none of this resolves your issues, please let us know and we can continue to help. Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On May 4, 2018, at 4:03 AM, Phil Hwrote: > > Hi all, > > I am trying to secure my NiFi installation. I have a client certificate > (nifi1.crt) and the CA for the intranet (ca.crt). I created the trust and > keystores as below: > > keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore > server_keystore.p12 -storetype PKCS12 > > keytool -import -file ca.crt -alias cacert -keystore truststore.jks > > And the relevant nifi.properties are set as follows > > nifi.security.keystore=./conf/server_keystore.p12 > nifi.security.keystoreType=PKCS12 > nifi.security.keystorePasswd= > nifi.security.keyPasswd= > nifi.security.truststore=./conf/truststore.jks > nifi.security.truststoreType=JKS > nifi.security.truststorePasswd= > > When I try and access the site via https, I receive the above error in > Firefox, and the following in the nifi-bootstrap.log (I have enabled > additional debugging). > > Using both of these certificates inside Apache httpd works on the client as > expected, so the certificates are fine. I have seen some references to > bugs/features in Jetty under Java 1.8 related to older TLS versions, but > I'm at a loss to explain this! Help!! > > Thanks, > Phil > > 2018-05-04 20:57:17,406 INFO [NiFi logging handler] org.apache.nifi.StdOut > Using SSLEngineImpl. > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Allow unsafe renegotiation: false > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Allow legacy hello messages: true > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Is initial handshake: true > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Is secure renegotiation: false > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 > for TLSv1 > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 > for TLSv1 > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1 > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 > for TLSv1 > 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut > Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for > TLSv1 > 2018-05-04 20:57:17,407 INFO [NiFi logging handler]
Re: ReplaceText Flow File Processing Count
I don't know the history of this particular processor, but I think the purpose of the session.get() with batches is similar to the concept of @SupportsBatching. Basically both of them should have better performance because you are handling multiple flow files in a single session. The supports batching concept is a bit more flexible as it is configurable by the user, where as this case is hard-coded into the processor. I suppose if there is some reason why you need to process 1 flow file at a time, you could set the back-pressure threshold to 1 on the queue leading into ReplaceText. On Fri, May 4, 2018 at 3:50 AM, Peter Wicks (pwicks)wrote: > Had a user notice today that a ReplaceText processor, scheduled to run every > 20 minutes, had processed all 14 files in queue at once. I looked at the code > and see that ReplaceText does not do a standard session.get, but instead > calls: > > final List flowFiles = > session.get(FlowFileFilters.newSizeBasedFilter(1, DataUnit.MB, 100)); > > Was there a design reason behind this? To us it was just really confusing > that we didn't have full control over how quickly FlowFile's move through > this processor. > > Thanks, > Peter
SSL_ERROR_NO_CYPHER_OVERLAP
Hi all, I am trying to secure my NiFi installation. I have a client certificate (nifi1.crt) and the CA for the intranet (ca.crt). I created the trust and keystores as below: keytool -import -trustcacerts -alias nifi1 -file nifi1.crt -keystore server_keystore.p12 -storetype PKCS12 keytool -import -file ca.crt -alias cacert -keystore truststore.jks And the relevant nifi.properties are set as follows nifi.security.keystore=./conf/server_keystore.p12 nifi.security.keystoreType=PKCS12 nifi.security.keystorePasswd= nifi.security.keyPasswd= nifi.security.truststore=./conf/truststore.jks nifi.security.truststoreType=JKS nifi.security.truststorePasswd= When I try and access the site via https, I receive the above error in Firefox, and the following in the nifi-bootstrap.log (I have enabled additional debugging). Using both of these certificates inside Apache httpd works on the client as expected, so the certificates are fine. I have seen some references to bugs/features in Jetty under Java 1.8 related to older TLS versions, but I'm at a loss to explain this! Help!! Thanks, Phil 2018-05-04 20:57:17,406 INFO [NiFi logging handler] org.apache.nifi.StdOut Using SSLEngineImpl. 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Allow unsafe renegotiation: false 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Allow legacy hello messages: true 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Is initial handshake: true 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Is secure renegotiation: false 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_GCM_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 for TLSv1 2018-05-04 20:57:17,407 INFO [NiFi logging handler] org.apache.nifi.StdOut
ReplaceText Flow File Processing Count
Had a user notice today that a ReplaceText processor, scheduled to run every 20 minutes, had processed all 14 files in queue at once. I looked at the code and see that ReplaceText does not do a standard session.get, but instead calls: final List flowFiles = session.get(FlowFileFilters.newSizeBasedFilter(1, DataUnit.MB, 100)); Was there a design reason behind this? To us it was just really confusing that we didn't have full control over how quickly FlowFile's move through this processor. Thanks, Peter
Re: GetMongoDB : How to pass parameters as input to GetMongoDB processor
As Bryan mentioned in the actual ‘GetSplunk’ thread, it is not available but it certainly makes sense to have that feature. If you’re interested in having that, please raise a Jira at https://issues.apache.org/jira - Sivaprasanna On Fri, 4 May 2018 at 12:56 PM, Brajendra Mishra < brajendra_mis...@persistent.com> wrote: > Hi Mike, > > Thanks a lot for your valuable inputs. > We tried GetMongoDB Template which you shared in previous mail with new > version of Apache NiFi (NiFi-1.6) , Its working fine and we are able to get > mongoDB data in desired format. > > One query: Is such input flow functionality available with 'GetSplunk' > processor too with NiFI 1.6.0? if yes, please share sample template or if > not, When can expect such implementation with 'GetSplunk' processor too? > > Brajendra Mishra > Persistent Systems Ltd. > > -Original Message- > From: Mike Thomsen> Sent: Thursday, May 03, 2018 6:56 PM > To: dev@nifi.apache.org > Subject: Re: GetMongoDB : How to pass parameters as input to GetMongoDB > processor > > Brajendra, > > I would recommend an update to 1.6.0. It'll make your life a lot easier on > this. I did that patch to GetMongo because I had a client that had an > explosion of GetMongos due to that inflexibility. With that said, *be aware > of this bug* in 1.6.0 w/ PutMongo if you use it and upgrade. It is fixed in > 1.7.0 (still in development): > > Migrating from 1.5.0 to 1.6.0 > >- PutMongo can fail in insert mode. Will be fixed in next release. In >the mean time you can set query keys for insert even though they'll be >ignored it should workaround the validation bug. > > > What it means is there is a validator function that is broken in PutMongo > when one is using the "insert mode" instead of "update mode." You can do a > work around by putting a dummy value in the "query key" field to make it > happy. > > On Thu, May 3, 2018 at 8:26 AM Pierre Villard > > wrote: > > > Hi, > > > > As Mike said: incoming relationship has been added for NiFi 1.6.0. > > https://issues.apache.org/jira/browse/NIFI-4827 > > > > Pierre > > > > 2018-05-03 14:09 GMT+02:00 Brajendra Mishra < > > brajendra_mis...@persistent.com > > >: > > > > > Hi Mike, > > > > > > I did attach the same in my previous mail. Well reattaching it again. > > > Well error is at GetMongoDB Processor and error text is : "Upstream > > > Connections is invalid because Processor does not allow upstream > > > connections but currently has 1" > > > > > > Brajendra Mishra > > > Persistent Systems Ltd. > > > > > > -Original Message- > > > From: Mike Thomsen > > > Sent: Thursday, May 03, 2018 5:20 PM > > > To: dev@nifi.apache.org > > > Subject: Re: GetMongoDB : How to pass parameters as input to > > > GetMongoDB processor > > > > > > Brajendra, > > > > > > Looks like the image didn't make it. > > > > > > On Wed, May 2, 2018 at 11:36 PM Brajendra Mishra < > > > brajendra_mis...@persistent.com> wrote: > > > > > > > Hi Mike, > > > > > > > > Thanks for responding. > > > > > > > > Here, I have attached missing image attachment. > > > > > > > > > > > > > > > > Brajendra Mishra > > > > > > > > Persistent Systems Ltd. > > > > > > > > > > > > > > > > *From:* Mike Thomsen > > > > *Sent:* Wednesday, May 02, 2018 6:24 PM > > > > > > > > > > > > *To:* dev@nifi.apache.org > > > > *Subject:* Re: GetMongoDB : How to pass parameters as input to > > > > GetMongoDB processor > > > > > > > > > > > > > > > > That might require 1.6.0. Also, your image didn't come through in > > > > your response to Sivaprasanna so resend that too. > > > > > > > > On Wed, May 2, 2018 at 8:37 AM Brajendra Mishra < > > > > brajendra_mis...@persistent.com> wrote: > > > > > > > > Hi Mike, > > > > > > > > Thanks a lot for responding. > > > > > > > > On your statement > > > > "That is its new default behavior if you leave the query field > > > > blank and have an incoming connection from another processor. That > > > > would be a good way to integrate the flow with another application" > > > > > > > > Could you please share a sample template for the same? > > > > > > > > > > > > Brajendra Mishra > > > > Persistent Systems Ltd. > > > > > > > > -Original Message- > > > > From: Mike Thomsen > > > > Sent: Wednesday, May 02, 2018 5:58 PM > > > > To: dev@nifi.apache.org > > > > Subject: Re: GetMongoDB : How to pass parameters as input to > > > > GetMongoDB processor > > > > > > > > GetMongo can also use the body of a flowfile for the query. That > > > > is its new default behavior if you leave the query field blank and > > > > have an incoming connection from another processor. That would be > > > > a good way to integrate the flow with another application. For > > > > example, you could add FetchKafka to the flow and have your > > > > applications post messages to Kafka with the queries they want it > > > > to run and FetchKafka
RE: GetMongoDB : How to pass parameters as input to GetMongoDB processor
Hi Mike, Thanks a lot for your valuable inputs. We tried GetMongoDB Template which you shared in previous mail with new version of Apache NiFi (NiFi-1.6) , Its working fine and we are able to get mongoDB data in desired format. One query: Is such input flow functionality available with 'GetSplunk' processor too with NiFI 1.6.0? if yes, please share sample template or if not, When can expect such implementation with 'GetSplunk' processor too? Brajendra Mishra Persistent Systems Ltd. -Original Message- From: Mike ThomsenSent: Thursday, May 03, 2018 6:56 PM To: dev@nifi.apache.org Subject: Re: GetMongoDB : How to pass parameters as input to GetMongoDB processor Brajendra, I would recommend an update to 1.6.0. It'll make your life a lot easier on this. I did that patch to GetMongo because I had a client that had an explosion of GetMongos due to that inflexibility. With that said, *be aware of this bug* in 1.6.0 w/ PutMongo if you use it and upgrade. It is fixed in 1.7.0 (still in development): Migrating from 1.5.0 to 1.6.0 - PutMongo can fail in insert mode. Will be fixed in next release. In the mean time you can set query keys for insert even though they'll be ignored it should workaround the validation bug. What it means is there is a validator function that is broken in PutMongo when one is using the "insert mode" instead of "update mode." You can do a work around by putting a dummy value in the "query key" field to make it happy. On Thu, May 3, 2018 at 8:26 AM Pierre Villard wrote: > Hi, > > As Mike said: incoming relationship has been added for NiFi 1.6.0. > https://issues.apache.org/jira/browse/NIFI-4827 > > Pierre > > 2018-05-03 14:09 GMT+02:00 Brajendra Mishra < > brajendra_mis...@persistent.com > >: > > > Hi Mike, > > > > I did attach the same in my previous mail. Well reattaching it again. > > Well error is at GetMongoDB Processor and error text is : "Upstream > > Connections is invalid because Processor does not allow upstream > > connections but currently has 1" > > > > Brajendra Mishra > > Persistent Systems Ltd. > > > > -Original Message- > > From: Mike Thomsen > > Sent: Thursday, May 03, 2018 5:20 PM > > To: dev@nifi.apache.org > > Subject: Re: GetMongoDB : How to pass parameters as input to > > GetMongoDB processor > > > > Brajendra, > > > > Looks like the image didn't make it. > > > > On Wed, May 2, 2018 at 11:36 PM Brajendra Mishra < > > brajendra_mis...@persistent.com> wrote: > > > > > Hi Mike, > > > > > > Thanks for responding. > > > > > > Here, I have attached missing image attachment. > > > > > > > > > > > > Brajendra Mishra > > > > > > Persistent Systems Ltd. > > > > > > > > > > > > *From:* Mike Thomsen > > > *Sent:* Wednesday, May 02, 2018 6:24 PM > > > > > > > > > *To:* dev@nifi.apache.org > > > *Subject:* Re: GetMongoDB : How to pass parameters as input to > > > GetMongoDB processor > > > > > > > > > > > > That might require 1.6.0. Also, your image didn't come through in > > > your response to Sivaprasanna so resend that too. > > > > > > On Wed, May 2, 2018 at 8:37 AM Brajendra Mishra < > > > brajendra_mis...@persistent.com> wrote: > > > > > > Hi Mike, > > > > > > Thanks a lot for responding. > > > > > > On your statement > > > "That is its new default behavior if you leave the query field > > > blank and have an incoming connection from another processor. That > > > would be a good way to integrate the flow with another application" > > > > > > Could you please share a sample template for the same? > > > > > > > > > Brajendra Mishra > > > Persistent Systems Ltd. > > > > > > -Original Message- > > > From: Mike Thomsen > > > Sent: Wednesday, May 02, 2018 5:58 PM > > > To: dev@nifi.apache.org > > > Subject: Re: GetMongoDB : How to pass parameters as input to > > > GetMongoDB processor > > > > > > GetMongo can also use the body of a flowfile for the query. That > > > is its new default behavior if you leave the query field blank and > > > have an incoming connection from another processor. That would be > > > a good way to integrate the flow with another application. For > > > example, you could add FetchKafka to the flow and have your > > > applications post messages to Kafka with the queries they want it > > > to run and FetchKafka would send that JSON to GetMongo as it comes > > > in. Or you could build a REST service that writes the JSON to disk and > > > use GetFile to load it. > > Lots of ways to do this. > > > > > > On Wed, May 2, 2018 at 6:42 AM Sivaprasanna > > > > > > wrote: > > > > > > > Since I'm not so sure about your exact use case, I have just > > > > created a rough template based on the simple example flow that I > > > > had posted earlier which is GenerateFlowfile -> UpdateAttribute > > > > -> GetMongo. I have attached the template