Benji, I've built a custom framework NAR [1] that has additional logging to identify which components, process groups, etc. are causing the issue. If you'd like, please feel free to save off your existing framework NAR and replace it with this, then share the relevant section of the log (matching the IDs to the components in your flow since it's sensitive and can't be shared). Depending on the structure of your flow (is it possible to share relevant screenshots?) that might help us identify, reproduce, and/or fix the issue.
I filed a Jira to add more logging [2], I'll take this patch and create a PR for it. Regards, Matt [1] https://drive.google.com/file/d/1f5xs0S2HPWIjTGKRuhCm6_LmxQ5XnbeD/view?usp=sharing [2] https://issues.apache.org/jira/browse/NIFI-10306 On Mon, Aug 1, 2022 at 12:46 PM BeNJ <intothev...@gmail.com> wrote: > > Updates: > Mark: Deleting the json allows nifi to start up(!!) - note that restarting > (with the newly generated json in place) causes the same issue on startup. > Joe: I'm happy to run custom debug nars to help figure out how to resolve > this. > > Thank you, I really appreciate everyone's help with this! > Benji > > > > On Mon, Aug 1, 2022 at 8:00 AM Joe Witt <joe.w...@gmail.com> wrote: >> >> Ben >> >> I didn't see any log levels that would help narrow this down. If Mark's >> suggestion does not address it it will likely require updating the flow >> configuration manually or a patched framework nar to do better logging. >> >> I'm also wondering if the json form of the flow could be imported on a clean >> nifi (you'd have to re-enter sensitive values) or imported to a registry >> then imported to nifi. >> >> Given you have hit this and someone else did (on stackoverflow) we clearly >> have an important case to figure out here as it is obviously disruptive. >> >> Thanks >> >> On Mon, Aug 1, 2022 at 5:41 AM Mark Payne <marka...@hotmail.com> wrote: >>> >>> Benji, >>> >>> I would recommend you try to remove (or rename) the flow.json.gz - but not >>> the flow.xml.gz. See if that makes any difference. >>> >>> Thanks >>> -Mark >>> >>> Sent from my iPhone >>> >>> On Jul 31, 2022, at 11:35 PM, BeNJ <intothev...@gmail.com> wrote: >>> >>> >>> Also please see the attached nifi.properties >>> >>> Thanks, >>> Benji >>> >>> On Sun, Jul 31, 2022 at 4:28 PM BeNJ <intothev...@gmail.com> wrote: >>>> >>>> Hi Joe, >>>> Stack with a couple of info logs from before and after, and the final exit >>>> shortly after. >>>> ------------------------------------------------------------------------------------------ >>>> 2022-07-31 16:20:35,311 INFO [main] >>>> o.a.n.g.StandardProcessGroupSynchronizer Added >>>> Connection[ID=cfee198f-3d2b-1513-f741-e71ad122aaaa, Source >>>> ID=cfee198e-3d2b-1513-7a9c-f0c2a8cf0d43, Dest >>>> ID=cfee19cf-3d2b-1513-4a87-5f50a90fdabf] to >>>> StandardProcessGroup[identifier=cfee1961-3d2b-1513-c8a6-fdf1a8fe4ff5,name=Add >>>> Customer User] >>>> 2022-07-31 16:20:35,311 INFO [main] >>>> o.a.n.g.StandardProcessGroupSynchronizer Added >>>> Connection[ID=cfee1974-3d2b-1513-e4df-9dbba1241682, Source >>>> ID=cfee1971-3d2b-1513-555c-1aedf0f0801f, Dest >>>> ID=cfee1970-3d2b-1513-de1d-f5bee9ad679e] to >>>> StandardProcessGroup[identifier=cfee1961-3d2b-1513-c8a6-fdf1a8fe4ff5,name=Add >>>> Customer User] >>>> 2022-07-31 16:20:35,317 INFO [Timer-Driven Process Thread-9] >>>> o.a.n.c.s.StandardControllerServiceNode Successfully enabled >>>> StandardControllerServiceNode[service=StandardOauth2AccessTokenProvider[id=cfee1b5b-3d2b-1513-7124-85b028901ac8], >>>> name=customer user management idp, active=true] >>>> 2022-07-31 16:20:35,325 INFO [main] o.a.n.c.s.AffectedComponentSet >>>> Starting the following components: AffectedComponentSet[inputPorts=[], >>>> outputPorts=[], remoteInputPorts=[], remoteOutputPorts=[], processors=[], >>>> controllerServices=[], reportingTasks=[]] >>>> 2022-07-31 16:20:35,328 WARN [main] org.eclipse.jetty.webapp.WebAppContext >>>> Failed startup of context >>>> o.e.j.w.WebAppContext@ffaaaf0{nifi-api,/nifi-api,file:///opt/nifi/nifi-current/work/jetty/nifi-web-api-1.16.0.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.16.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.16.0.war} >>>> org.apache.nifi.controller.serialization.FlowSynchronizationException: >>>> java.lang.IllegalStateException: Cannot add Connection to Process Group >>>> because source is an Output Port that does not belong to a child Process >>>> Group >>>> at >>>> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:362) >>>> at >>>> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:185) >>>> at >>>> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:43) >>>> at >>>> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1479) >>>> at >>>> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104) >>>> at >>>> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:815) >>>> at >>>> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:538) >>>> at >>>> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073) >>>> at >>>> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002) >>>> at >>>> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) >>>> at >>>> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) >>>> at >>>> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) >>>> at >>>> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916) >>>> at >>>> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) >>>> at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at org.eclipse.jetty.server.Server.start(Server.java:423) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at org.eclipse.jetty.server.Server.doStart(Server.java:387) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1008) >>>> at org.apache.nifi.NiFi.<init>(NiFi.java:170) >>>> at org.apache.nifi.NiFi.<init>(NiFi.java:82) >>>> at org.apache.nifi.NiFi.main(NiFi.java:330) >>>> Caused by: java.lang.IllegalStateException: Cannot add Connection to >>>> Process Group because source is an Output Port that does not belong to a >>>> child Process Group >>>> at >>>> org.apache.nifi.groups.StandardProcessGroup.addConnection(StandardProcessGroup.java:1154) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addConnection(StandardProcessGroupSynchronizer.java:1853) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeConnections(StandardProcessGroupSynchronizer.java:531) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:388) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.lambda$synchronize$0(StandardProcessGroupSynchronizer.java:221) >>>> at >>>> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:462) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:219) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3830) >>>> at >>>> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:353) >>>> ... 45 common frames omitted >>>> 2022-07-31 16:20:35,552 INFO [Timer-Driven Process Thread-3] >>>> o.a.n.c.s.StandardControllerServiceNode Successfully enabled >>>> StandardControllerServiceNode[service=MongoDBControllerService[id=19573a5c-3f9e-3cdc-b80c-1990fdd33c69], >>>> name=MongoDB CR - Main, active=true] >>>> ... >>>> ... >>>> 2022-07-31 16:20:35,733 INFO [main] org.eclipse.jetty.server.Server >>>> Started @26964ms >>>> 2022-07-31 16:20:35,734 WARN [main] org.apache.nifi.web.server.JettyServer >>>> Failed to start web server... shutting down. >>>> org.apache.nifi.controller.serialization.FlowSynchronizationException: >>>> java.lang.IllegalStateException: Cannot add Connection to Process Group >>>> because source is an Output Port that does not belong to a child Process >>>> Group >>>> at >>>> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:362) >>>> at >>>> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:185) >>>> at >>>> org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:43) >>>> at >>>> org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1479) >>>> at >>>> org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104) >>>> at >>>> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:815) >>>> at >>>> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:538) >>>> at >>>> org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073) >>>> at >>>> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002) >>>> at >>>> org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746) >>>> at >>>> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379) >>>> at >>>> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449) >>>> at >>>> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414) >>>> at >>>> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916) >>>> at >>>> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288) >>>> at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:426) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169) >>>> at org.eclipse.jetty.server.Server.start(Server.java:423) >>>> at >>>> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110) >>>> at >>>> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97) >>>> at org.eclipse.jetty.server.Server.doStart(Server.java:387) >>>> at >>>> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73) >>>> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1008) >>>> at org.apache.nifi.NiFi.<init>(NiFi.java:170) >>>> at org.apache.nifi.NiFi.<init>(NiFi.java:82) >>>> at org.apache.nifi.NiFi.main(NiFi.java:330) >>>> Caused by: java.lang.IllegalStateException: Cannot add Connection to >>>> Process Group because source is an Output Port that does not belong to a >>>> child Process Group >>>> at >>>> org.apache.nifi.groups.StandardProcessGroup.addConnection(StandardProcessGroup.java:1154) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addConnection(StandardProcessGroupSynchronizer.java:1853) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeConnections(StandardProcessGroupSynchronizer.java:531) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:388) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.addProcessGroup(StandardProcessGroupSynchronizer.java:1019) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronizeChildGroups(StandardProcessGroupSynchronizer.java:423) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:373) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.lambda$synchronize$0(StandardProcessGroupSynchronizer.java:221) >>>> at >>>> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:462) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:219) >>>> at >>>> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3830) >>>> at >>>> org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:353) >>>> ... 45 common frames omitted >>>> 2022-07-31 16:20:35,738 INFO [Thread-1] org.apache.nifi.NiFi Application >>>> Server shutdown started >>>> ------------------------------------------------------------------------------------------ >>>> >>>> David: >>>> df -h for this fs: >>>> /dev/mapper/ubuntu--vg-ubuntu--lv 97G 41G 52G 44% / >>>> I'll be able to share the nifi.properties in a few hours when I'm back, >>>> please let me know if there's anything else I can share or enable. >>>> >>>> Thanks, >>>> Benji >>>> >>>> On Sun, Jul 31, 2022 at 3:37 PM Joe Witt <joe.w...@gmail.com> wrote: >>>>> >>>>> Benji >>>>> >>>>> What would be helpful to see is the full stacktrace like shown in this >>>>> stackoverflow post >>>>> https://stackoverflow.com/questions/72459434/apache-nifi-1-16-0-windows-server-closes-immediately >>>>> >>>>> This class changes often so I want to make sure we check the right one >>>>> for your version. >>>>> >>>>> Thanks >>>>> >>>>> On Sun, Jul 31, 2022 at 1:25 PM David Early via users >>>>> <users@nifi.apache.org> wrote: >>>>>> >>>>>> Benji, >>>>>> >>>>>> Can you share your nifi.properties file? >>>>>> >>>>>> This problem can show up if the disk fills up but depends on the >>>>>> configuration. >>>>>> >>>>>> Also, what does df -h show for your disks (assuming you are running on >>>>>> Linux)? >>>>>> >>>>>> Dave >>>>>> >>>>>> >>>>>> On Sun, Jul 31, 2022, 2:21 PM Benji Benning <intothev...@gmail.com> >>>>>> wrote: >>>>>>> >>>>>>> Not really around the below error. >>>>>>> Anything I can move to debug or trace in logback.xml to add some color >>>>>>> around this? >>>>>>> >>>>>>> Thanks, >>>>>>> Benji >>>>>>> >>>>>>> Sent from my iPhone >>>>>>> >>>>>>> On Jul 31, 2022, at 15:12, Joe Witt <joe.w...@gmail.com> wrote: >>>>>>> >>>>>>> >>>>>>> Benji >>>>>>> >>>>>>> Is there more with the logs like component uuids and such? >>>>>>> >>>>>>> Thanks >>>>>>> >>>>>>> On Sun, Jul 31, 2022 at 1:08 PM Benji Benning <intothev...@gmail.com> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hi Joe, >>>>>>>> Unfortunately it does indeed contain sensitive info. >>>>>>>> Is there any debug info I can provide? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Benji >>>>>>>> >>>>>>>> Sent from my iPhone >>>>>>>> >>>>>>>> On Jul 31, 2022, at 12:25, Joe Witt <joe.w...@gmail.com> wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Can you share the flow or does it contain sensitve info? >>>>>>>> >>>>>>>> On Sun, Jul 31, 2022 at 10:01 AM BeNJ <intothev...@gmail.com> wrote: >>>>>>>>> >>>>>>>>> Hi, >>>>>>>>> I'm a little at a loss so I figured maybe someone here would have an >>>>>>>>> idea of how to get out of this situation. >>>>>>>>> My NiFI is crashing on startup with the following: >>>>>>>>> -------------------------------------- >>>>>>>>> 2022-07-31 03:10:05,448 INFO [main] o.a.n.c.s.AffectedComponentSet >>>>>>>>> Starting the following components: >>>>>>>>> AffectedComponentSet[inputPorts=[], outputPorts=[], >>>>>>>>> remoteInputPorts=[], remoteOutputPorts=[], processors=[], >>>>>>>>> controllerServices=[], reportingTasks=[]] >>>>>>>>> 2022-07-31 03:10:05,452 WARN [main] >>>>>>>>> org.eclipse.jetty.webapp.WebAppContext Failed startup of context >>>>>>>>> o.e.j.w.WebAppContext@3ed03652{nifi-api,/nifi-api,file:///opt/nifi/nifi-current/work/jetty/nifi-web-api-1.16.0.war/webapp/,UNAVAILABLE}{./work/nar/extensions/nifi-server-nar-1.16.0.nar-unpacked/NAR-INF/bundled-dependencies/nifi-web-api-1.16.0.war} >>>>>>>>> org.apache.nifi.controller.serialization.FlowSynchronizationException: >>>>>>>>> java.lang.IllegalStateException: Cannot add Connection to Process >>>>>>>>> Group because source is an Output Port that does not belong to a >>>>>>>>> child Process Group >>>>>>>>> -------------------------------------- >>>>>>>>> I only restarted because out of nowhere i started seeing the >>>>>>>>> following errors repeating: >>>>>>>>> Processors cannot write to Content Repo: Unable to write flowfile >>>>>>>>> content to content repository container default due to archive file >>>>>>>>> size constraints; waiting for archive cleanup >>>>>>>>> >>>>>>>>> To resolve this issue, someone said online to set >>>>>>>>> nifi.content.repository.archive.enabled to false. I restarted and >>>>>>>>> nifi wouldn't come up again. >>>>>>>>> >>>>>>>>> I've tried reverting the flow.xml.gz to the oldest one in the archive >>>>>>>>> folder (was from 7pm yesterday), no luck. >>>>>>>>> it's a large config (1000+ active processors). >>>>>>>>> I would try to disable / delete the offending output port, but I >>>>>>>>> don't see how to find it from the logs. >>>>>>>>> >>>>>>>>> Any direction would be greatly appreciated. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Benji
