Dan, I was not able to reproduce the issue you are encountering — I built NiFi 0.7.0 and deployed with a vanilla (HTTP) configuration and verified everything worked. I then changed only the relevant nifi.properties settings to secure the instance and was still able to bring it up without a problem. I navigated to https://localhost:8443/nifi <https://localhost:8443/nifi> and was able to provide a client certificate which was recognized and the app loaded. This leads me to believe it is a configuration issue with the machine that it is running on. You provided quite an extensive list of debugging activities and configuration awareness in your initial email, which I appreciate.
How are these machines provisioned? Are they bare-metal or VM/containers? Is it possible to deploy a different “production” instance and verify that the issue is reproducible there? Could it be something weird with IPv4 vs. IPv6? Can you try binding to “::” as in “nifi.web.https.host=::”? Can you run “lsof -i :8443” or “nmap -p 8443 127.0.0.1”? You should see output like below if the service is running: hw12203:/Users/alopresto/Workspace/scratch/latest_conf (master) alopresto 🔓 2578s @ 16:16:40 $ lsof -i :8443 COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME ... java 16426 alopresto 1621u IPv4 0x84a462b37a5b879 0t0 TCP *:pcsync-https (LISTEN) hw12203:/Users/alopresto/Workspace/scratch/latest_conf (master) alopresto 🔓 2799s @ 16:20:21 $ nmap -p 8443 127.0.0.1 Starting Nmap 7.31 ( https://nmap.org ) at 2017-07-26 16:20 PDT Nmap scan report for nifi.nifi.apache.org (127.0.0.1) Host is up (0.00045s latency). PORT STATE SERVICE 8443/tcp open https-alt Nmap done: 1 IP address (1 host up) scanned in 0.05 seconds Here are the relevant sections from my nifi-app.log on startup. # Plain 2017-07-26 15:32:36,053 INFO [main] org.eclipse.jetty.server.ServerConnector Started ServerConnector@6e664075{HTTP/1.1}{0.0.0.0:8080} 2017-07-26 15:32:36,053 INFO [main] org.eclipse.jetty.server.Server Started @27339ms 2017-07-26 15:32:36,525 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: 2017-07-26 15:32:36,525 INFO [main] org.apache.nifi.web.server.JettyServer http://192.168.1.12:8080/nifi 2017-07-26 15:32:36,525 INFO [main] org.apache.nifi.web.server.JettyServer http://<external_IP>:8080/nifi 2017-07-26 15:32:36,526 INFO [main] org.apache.nifi.web.server.JettyServer http://127.0.0.1:8080/nifi 2017-07-26 15:32:36,527 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap 2017-07-26 15:32:36,527 INFO [main] org.apache.nifi.NiFi Controller initialization took 6865150481 nanoseconds. # TLS 2017-07-26 15:59:56,451 INFO [main] org.eclipse.jetty.server.ServerConnector Started ServerConnector@6e466fdf{SSL-http/1.1}{0.0.0.0:8443} 2017-07-26 15:59:56,452 INFO [main] org.eclipse.jetty.server.Server Started @25966ms 2017-07-26 15:59:56,887 INFO [main] org.apache.nifi.web.server.JettyServer NiFi has started. The UI is available at the following URLs: 2017-07-26 15:59:56,887 INFO [main] org.apache.nifi.web.server.JettyServer https://0.0.0.0:8443/nifi 2017-07-26 15:59:56,888 INFO [main] org.apache.nifi.BootstrapListener Successfully initiated communication with Bootstrap 2017-07-26 15:59:56,889 INFO [main] org.apache.nifi.NiFi Controller initialization took 6433250661 nanoseconds. Last but not least, as I have done this and similar things many times, are you sure that the proper sections are correctly commented out/enabled in your nifi.properties when you try to start up? I can’t think of a scenario off the top of my head that would cause your issue, but perhaps if there are conflicting instructions on hostname and port? Andy LoPresto alopre...@apache.org alopresto.apa...@gmail.com PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > On Jul 26, 2017, at 3:29 PM, Andy LoPresto <alopre...@apache.org> wrote: > > Definitely recommend you upgrade if at all possible — 0.7.4 has a number of > fixes over 0.7.0 (which is over a year old now) [1]. If you can migrate to > the 1.x line, which sees much more active development and had a large number > of framework changes, I think you would get even more value. > > Either way, we should still be able to diagnose the problem in 0.7.0 and at > least maintain a known issue for other users if necessary. > > [1] > https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.7.4 > > <https://cwiki.apache.org/confluence/display/NIFI/Release+Notes#ReleaseNotes-Version0.7.4> > > > Andy LoPresto > alopre...@apache.org <mailto:alopre...@apache.org> > alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> > PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 > >> On Jul 26, 2017, at 12:19 PM, Dan Morris <dgmorri...@gmail.com >> <mailto:dgmorri...@gmail.com>> wrote: >> >> Andy, >> >> Thanks for taking a look… >> >> May end up just needing to uninstall/reinstall and see if the same issues >> happen again. I may trying upgrading to a newer version of nifi as well. >> >> Again…appreciate you all taking a look. >> >> Thanks, >> Dan >> >> >> From: Andy LoPresto <alopre...@apache.org <mailto:alopre...@apache.org>> >> Reply-To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> >> Date: Wednesday, July 26, 2017 at 3:17 PM >> To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> >> Subject: Re: NiFi UI Not Starting >> >> Dan, >> >> Sorry we are not more helpful on this. Seems to be an extremely unusual >> circumstance. I would suggest modifying bootstrap.conf to enable remote >> debugging and use your IDE to step through the execution of the Jetty code. >> Something about the logic path is different when creating the HTTPS >> connector — either the context factory is not getting formed correctly, the >> network interfaces are not being enumerated, or something is violating an >> external permission/policy block. I will try to reproduce this locally as >> well but I have not encountered this before when setting up a secure 0.x >> instance. >> >> >> Andy LoPresto >> alopre...@apache.org <mailto:alopre...@apache.org> >> alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> >> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >> >>> On Jul 26, 2017, at 11:37 AM, Dan Morris <dgmorri...@gmail.com >>> <mailto:dgmorri...@gmail.com>> wrote: >>> >>> Hi Andy, >>> >>> It’s the same instance of nifi… in our nifi.properties file, we just >>> comment/uncomment the “Disable TLS” or “Enable TLS” sections depending on >>> which “mode” we want nifi to run in. >>> >>> When we comment out the “Enable TLS” section and uncomment the “Disable >>> TLS” sections, the the UI binds to both localhost and the IPv4 Address (the >>> only other eth device). >>> >>> Thanks, >>> Dan Morris >>> Mobile: 443-992-2848 >>> GV: 410-861-0206 >>> >>> >>> From: Andy LoPresto <alopre...@apache.org <mailto:alopre...@apache.org>> >>> Reply-To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> >>> Date: Wednesday, July 26, 2017 at 2:30 PM >>> To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> >>> Subject: Re: NiFi UI Not Starting >>> >>> Dan, >>> >>> You said that if you run an unsecured instance of NiFi on the production >>> server, it starts successfully? What host(s) does it bind to in that case? >>> >>> Andy LoPresto >>> alopre...@apache.org <mailto:alopre...@apache.org> >>> alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> >>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>> >>>> On Jul 26, 2017, at 11:21 AM, Andy LoPresto <alopre...@apache.org >>>> <mailto:alopre...@apache.org>> wrote: >>>> >>>> I have to refamiliarize myself with 0.7.0 as it’s a bit of an older >>>> version, but the code we should be looking at is [1] and [2]. >>>> >>>> [1] >>>> https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L557 >>>> >>>> <https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L557> >>>> [2] >>>> https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L797 >>>> >>>> <https://github.com/apache/nifi/blob/rel/nifi-0.7.0/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-jetty/src/main/java/org/apache/nifi/web/server/JettyServer.java#L797> >>>> >>>> >>>> Andy LoPresto >>>> alopre...@apache.org <mailto:alopre...@apache.org> >>>> alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> >>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>> >>>>> On Jul 26, 2017, at 11:15 AM, Dan Morris <dgmorri...@gmail.com >>>>> <mailto:dgmorri...@gmail.com>> wrote: >>>>> >>>>> We tried: >>>>> >>>>> nifi.web.https.host=localhost >>>>> >>>>> nifi.web.https.host= >>>>> >>>>> nifi.web.https.host=0.0.0.0 >>>>> >>>>> no impact, UI still would not bind to HTTPS port. >>>>> >>>>> We decided to leave it at 0.0.0.0 as that’s the recommended configuration >>>>> in the Administrators guide for binding to all interfaces. >>>>> >>>>> Thanks, >>>>> Dan Morris >>>>> Mobile: 443-992-2848 >>>>> GV: 410-861-0206 >>>>> >>>>> >>>>> From: Andy LoPresto <alopre...@apache.org <mailto:alopre...@apache.org>> >>>>> Reply-To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> >>>>> Date: Wednesday, July 26, 2017 at 1:56 PM >>>>> To: <users@nifi.apache.org <mailto:users@nifi.apache.org>> >>>>> Subject: Re: NiFi UI Not Starting >>>>> >>>>> Dan, >>>>> >>>>> I am wondering if it is an issue with binding to 0.0.0.0 — are there any >>>>> differences between the test and production server non-NiFi >>>>> configurations that would prevent this? Can you try setting >>>>> nifi.web.https.host=localhost instead? >>>>> >>>>> >>>>> Andy LoPresto >>>>> alopre...@apache.org <mailto:alopre...@apache.org> >>>>> alopresto.apa...@gmail.com <mailto:alopresto.apa...@gmail.com> >>>>> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4 BACE 3C6E F65B 2F7D EF69 >>>>> >>>>>> On Jul 26, 2017, at 10:44 AM, Dan Morris <dgmorri...@gmail.com >>>>>> <mailto:dgmorri...@gmail.com>> wrote: >>>>>> >>>>>> Hi Joe, >>>>>> >>>>>> I’ve attached relevant files… tried to redact sensitive info… hope I >>>>>> didn’t cut too much from the logs… >>>>>> >>>>>> Thanks, >>>>>> Dan >>>>>> >>>>>> >>>>>> >>>>>> On 7/26/17, 9:30 AM, "Joe Witt" <joe.w...@gmail.com >>>>>> <mailto:joe.w...@gmail.com>> wrote: >>>>>> >>>>>> Dan - are you able to share the nifi-app and nifi-bootstrap logs? >>>>>> >>>>>> Thanks >>>>>> >>>>>> On Wed, Jul 26, 2017 at 9:21 AM, Dan Morris <dgmorri...@gmail.com >>>>>> <mailto:dgmorri...@gmail.com>> wrote: >>>>>> >>>>>> >>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> >>>>>>> >>>>>>> I’m having an issue getting NiFi to start correctly. Here’s my >>>>>>> situation: >>>>>>> >>>>>>> I’m currently running v0.7.0. >>>>>>> I have a production server and a test server, with identical >>>>>>> configurations >>>>>>> (OS, Java, Java security config, nifi versions, nifi configs, >>>>>>> keystores/truststores, etc). >>>>>>> When I run nifi via normal HTTP (e.g. no security) in both Prod/Test >>>>>>> they >>>>>>> both start & load the UI as expected. >>>>>>> When I run nifi via HTTPs (e.g. security settings) the Test server >>>>>>> starts >>>>>>> and loads UI as expected. >>>>>>> However, on the Prod system, I receive the following error and java >>>>>>> does not >>>>>>> bind to 8443: >>>>>>> >>>>>>> >>>>>>> >>>>>>> 2017-07-25 16:30:51,346 WARN [main] >>>>>>> org.apache.nifi.web.server.JettyServer >>>>>>> NiFi has started, but the UI is not available on any hosts. Please >>>>>>> verify >>>>>>> the host properties. >>>>>>> >>>>>>> >>>>>>> >>>>>>> I reviewed the source code and it looks like this error is logged when >>>>>>> the >>>>>>> “URLs” is empty. >>>>>>> Here is what I *think* are the relevant properties from my nifi config, >>>>>>> when >>>>>>> trying to start up using TLS (running on both Prod & Test), again, Test >>>>>>> starts fine, Prod throws the error above. >>>>>>> >>>>>>> >>>>>>> >>>>>>> # Enable TLS >>>>>>> >>>>>>> nifi.web.http.host= >>>>>>> >>>>>>> nifi.web.https.host=0.0.0.0 >>>>>>> >>>>>>> nifi.web.http.port= >>>>>>> >>>>>>> nifi.web.https.port=8443 >>>>>>> >>>>>>> nifi.security.keystore=<path_to_keystore> >>>>>>> >>>>>>> nifi.security.keystoreType=JKS >>>>>>> >>>>>>> nifi.security.keystorePasswd=<keystore_password> >>>>>>> >>>>>>> nifi.security.keyPasswd=<key_password> >>>>>>> >>>>>>> nifi.security.truststore=<path_to_trust_store> >>>>>>> >>>>>>> nifi.security.truststoreType=JKS >>>>>>> >>>>>>> nifi.security.truststorePasswd=<trust_store_password> >>>>>>> >>>>>>> nifi.security.needClientAuth=true >>>>>>> >>>>>>> >>>>>>> >>>>>>> Again, I’ve manually validated the correct paths, correct passwords to >>>>>>> JKS >>>>>>> files., etc. >>>>>>> I’ve verified that there are no other processes binding to 8443 possibly >>>>>>> blocking nifi from the port. >>>>>>> I’ve tried changing the port number (e.g. to 8445), no effect >>>>>>> I’ve turned off IPTables. >>>>>>> Generally, I run nifi as a “nifi” user, however, I’ve also tried >>>>>>> running it >>>>>>> as root to see if that had an effect of allowing the UI on Prod to >>>>>>> start…no >>>>>>> impact. >>>>>>> I’ve tried also starting up nifi with a blank/default flow file, no >>>>>>> effect. >>>>>>> >>>>>>> >>>>>>> >>>>>>> Any thoughts/suggestions on what I can do next, short of uninstalling >>>>>>> nifi >>>>>>> and reinstalling? >>>>>>> >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Dan >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> <bootstrap.conf><nifi-app.log><nifi-bootstrap.log><nifi.properties> >>>>> >>>>> >>>> >>>> >>> >>> >> >> >
signature.asc
Description: Message signed with OpenPGP using GPGMail
