Rick, Can you confirm the certificate has a chain of trust with the default JDK trusted certs? (i.e. trusted by the JVM)
Cheers On Mon, Nov 7, 2016 at 3:38 AM, Ricky Saltzer <ri...@cloudera.com> wrote: > Hey Andy - > > Thanks again for the help. > > The error message seems indicative that it doesn't seem to properly read > the keystore file. One thing to note, if I point the nifi properties to a > bogus keystore location, then it actually throws a FileNotFound exception. > This is really odd behavior, because as I mentioned I'm able to start it in > standalone mode using the correct keystore location, just as I try to do in > clustered mode. > > I've attached both the clustered [1] nifi.properties, which doesn't work, > and the standalone [2] which does work. . I restored it to a more basic > configuration without the encrypted configuration, but with SSL still > enabled. I also added a diff [3] of both the standalone and clustered > properties file. Note that I I only have NiFi configured to use the > keystore and not a truststore. I've redacted a few of the values in the > property files, but be assured that the keystore is most definitely valid > and is readable / locatable, as starting in standalone works just fine. > > I ran the SSL command [4] you gave me, minus the three PEM file arguments > as I don't have any of those on hand. I hope that is fine. The output still > looks good. > > [1] https://gist.github.com/rickysaltzer/712aa6586592fe6628db2d57cec7a562 > [2] https://gist.github.com/rickysaltzer/fe11c8233e4434eacedd7fd0a083d950 > [3] https://gist.github.com/rickysaltzer/d715c7451eb554a54f14ec6e64da8558 > [4] https://gist.github.com/rickysaltzer/5d7cdeff8868bfc1f47010189735411a > > > > > On Fri, Nov 4, 2016 at 7:48 PM, Andy LoPresto <alopre...@apache.org> > wrote: > > > Hi Ricky, > > > > Sorry, should have noted that the debug output goes to > nifi-bootstrap.log, > > so thanks Mark for jumping in to help there. > > > > If you look at the top of that log, you’ll note that there is no keystore > > file provided and the truststore loaded is the default JRE cacerts > > truststore. Can you please provide your nifi.properties file in a Gist, > **taking > > care to redact any sensitive values** like keystore/truststore passwords, > > although I think from looking at your log output, you are taking > advantage > > of the encrypted configuration feature, so even viewing the encrypted > > values should be ok. Could you also please provide the directory listing > > where the keystore and truststore are located including the permissions > and > > ownership information? > > > > There may be a bug in the logic between cluster and standalone mode, but > I > > haven’t encountered this behavior before. If you can start NiFi in > > standalone mode, could you please provide the output of the following > > command run from the terminal? It will simulate an HTTPS connection to > the > > server and verify the key and certificate presented by NiFi. > > > > * host — the NiFi hostname > > * port — the port NiFi is running on > > * path_to_your_cert.pem — the public key certificate identifying the > > client/user (i.e. what you load into your browser to authenticate) > > * path_to_your_key.key — the private key identifying the client/user > > * path_to_your_CA_cert.pem — the public key certificate identifying the > CA > > which signed your NiFi server certificate (if self-signed, provide that > > certificate) > > > > $ openssl s_client -connect <host:port> -debug -state -cert > > <path_to_your_cert.pem> -key <path_to_your_key.pem> -CAfile > > <path_to_your_CA_cert.pem> > > > > Andy LoPresto > > alopre...@apache.org > > *alopresto.apa...@gmail.com <alopresto.apa...@gmail.com>* > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > On Nov 4, 2016, at 11:21 AM, Ricky Saltzer <ri...@cloudera.com> wrote: > > > > Hey guys - > > > > I went ahead and uploaded the boostrap log. I took a look at it and it > > seems to be the same error [1] > > > > [1]: > > https://gist.githubusercontent.com/rickysaltzer/ > > b156594f92066873f80928eddf84e7bb/raw/4d0e018038b332f4fdf14644910dfa > > 9e70c57e49/gistfile1.txt > > > > On Fri, Nov 4, 2016 at 2:14 PM, Mark Payne <marka...@hotmail.com> wrote: > > > > Hey Ricky, > > > > When you enable debug logging for SSL, it writes to StdErr (or StdOut?) > so > > it will end up in your logs/nifi-bootstrap.log instead of nifi-app.log. > > Can you give that a look? > > > > Thanks > > -Mark > > > > On Nov 4, 2016, at 2:07 PM, Ricky Saltzer <ri...@cloudera.com> wrote: > > > > Hey Andy - > > > > Thanks for the response. I'm currently just trying to get one node in > > clustered mode before adding a second. The keystore is stored locally and > > I've confirmed it's readable, as it was able to start once I took it out > > > > of > > > > clustered mode. I added that line to the bootstrap.conf, but I don't > > believe any additional logging was produced in regards to troubleshooting > > this problem. Just in case, I've attached the entire log [1]. > > > > [1]: > > https://gist.githubusercontent.com/rickysaltzer/ > > > > ed454d87d2207d5acab401a473d4be57/raw/425c42da762fc5cc997153d48b09f0 > > fedabc88bb/gistfile1.txt <https://gist.githubusercontent.com/ > rickysaltzer/ > > ed454d87d2207d5acab401a473d4be57/raw/425c42da762fc5cc997153d48b09f0 > > fedabc88bb/gistfile1.txt> > > > > > > On Wed, Nov 2, 2016 at 7:08 PM, Andy LoPresto <alopre...@apache.org > > > > <mailto:alopre...@apache.org>> wrote: > > > > > > Hi Ricky, > > > > Sorry to hear you are having this issue. Is the keystore available on > > > > all > > > > nodes of the cluster? It appears from the log message that the keystore > > > > is > > > > not found during startup. To further debug, you can add the following > > > > line > > > > in bootstrap.conf to provide additional logging: > > > > java.arg.15=-Djavax.net.debug=ssl,handshake > > > > Andy LoPresto > > alopre...@apache.org <mailto:alopre...@apache.org> > > *alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> < > > > > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com>>* > > > > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > > > > On Nov 2, 2016, at 2:25 PM, Ricky Saltzer <ri...@cloudera.com> wrote: > > > > Hey all - > > > > I'm using NiFi 1.0 and I'm having an issue using secure mode with a > > > > local > > > > key store while in clustered mode. If I set the node in clustered mode, > > > > and > > > > also provide a valid keystore, I receive a KeyStoreException [1]. If I > > > > set > > > > the configuration to not use clustered mode, NiFi will start up fine > > > > with > > > > the provided key store. Am I supposed to be storing this key store in > > Zookeeper somewhere? > > > > > > [1] > > > > > > Caused by: java.security.KeyStoreException: not found > > > > > > at java.security.KeyStore.getInstance(KeyStore.java:839) > > ~[na:1.8.0_11] > > > > at > > org.apache.nifi.io.socket.SSLContextFactory.<init>( > > SSLContextFactory.java:61) > > ~[nifi-socket-utils-1.0.0.jar:1.0.0] > > > > at > > org.apache.nifi.cluster.protocol.spring.ServerSocketConfigurationFacto > > ryBean.getObject(ServerSocketConfigurationFactoryBean.java:45) > > ~[nifi-framework-cluster-protocol-1.0.0.jar:1.0.0] > > > > at > > org.apache.nifi.cluster.protocol.spring.ServerSocketConfigurationFacto > > ryBean.getObject(ServerSocketConfigurationFactoryBean.java:30) > > ~[nifi-framework-cluster-protocol-1.0.0.jar:1.0.0] > > > > at > > org.springframework.beans.factory.support.FactoryBeanRegistrySupport. > > doGetObjectFromFactoryBean(FactoryBeanRegistrySupport.java:168) > > ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] > > > > ... 69 common frames omitted > > > > Caused by: java.security.NoSuchAlgorithmException: KeyStore not > > > > available > > > > > > at sun.security.jca.GetInstance.getInstance(GetInstance.java:159) > > ~[na:1.8.0_11] > > > > at java.security.Security.getImpl(Security.java:695) > > > > ~[na:1.8.0_11] > > > > > > at java.security.KeyStore.getInstance(KeyStore.java:836) > > ~[na:1.8.0_11] > > > > ... 73 common frames omitted > > > > > > > > > > > > -- > > Ricky Saltzer > > http://www.cloudera.com <http://www.cloudera.com/> > > > > > > > > > > > > -- > > Ricky Saltzer > > http://www.cloudera.com > > > > > > > > > -- > Ricky Saltzer > http://www.cloudera.com >
